Microsoft® SQL Server 2005 Notification Services

Next Page

Book Cover
Microsoft® SQL Server 2005 Notification Services
By Shyam Pather
...............................................
Publisher: Sams
Pub Date: February 14, 2006
Print ISBN-10: 0-672-32779-1
Print ISBN-13: 978-0-672-32779-7
Pages: 720

Table of Contents | Index
Overview

Microsoft SQL Server 2005 Notification Services covers the basic problems of notification applications, describes the SQL Notification Services platform and describes how to use it to build rich, scalable notification applications. Written by Shyam Pather, the Technical Lead for the SQL Notification Services Team, this book is based on his three years of experience teaching SQL Notification Services to new customers. This book is your complete resource for learning SQL Notification Services application development and the underlying architectural concepts.

Next Page


Table of Contents

Previous Page
Next Page

Book Cover
Microsoft® SQL Server 2005 Notification Services
By Shyam Pather
...............................................
Publisher: Sams
Pub Date: February 14, 2006
Print ISBN-10: 0-672-32779-1
Print ISBN-13: 978-0-672-32779-7
Pages: 720

Table of Contents | Index

Copyright
About the Author
Acknowledgments
We Want to Hear from You!
Introduction
 | Who Should Read This Book
 | Teaching Approach
 | Organization of This Book
 | About the CD-ROM
 | Conventions
Part I: Notification Services Concepts
 | Chapter 1. An Overview of Notification Applications
 |  | The Publish-Subscribe Phenomenon
 |  | Components of a Pub-Sub System
 |  | Challenges in Building Notification Applications
 |  | SQL Server 2005 Notification Services
 |  | Summary
 | Chapter 2. Getting Set Up
 |  | Your Development Environment: The Big Picture
 |  | Prerequisites and System Requirements
 |  | Standard or Enterprise Edition?
 |  | Setting Up Your Development Environment
 | Chapter 3. The Simplest Notification Application: Stock Quotes
 |  | The SQL-NS Application Model
 |  | Building the Stock Application's ADF
 |  | Specifying Other Parts of the Stock Application
 |  | Running the Stock Application
 |  | Inside the Running Application
 |  | What Has the SQL-NS Platform Provided?
 |  | Cleaning Up the Instance and Application
 |  | Summary
 | Chapter 4. Working with SQL-NS Instances
 |  | What Is an Instance?
 |  | Creating an Instance and Making It Run
 |  | Examining the Parts of the Instance
 |  | Deployment of SQL-NS Instances on Multiple Servers
 |  | The Instance Configuration File in Detail
 |  | Cleaning Up the Instance
 |  | Summary
Part II: Notification Services Application Development
 | Chapter 5. Designing and Prototyping an Application
 |  | What Is a SQL-NS Application?
 |  | Stages in the Application Development Process
 |  | Supporting the Sample Application
 |  | Initial Design: Seven Important Questions About Your Application
 |  | Building the Application Prototype
 |  | Summary
 | Chapter 6. Completing the Application Prototype: Scheduled Subscriptions and Application State
 |  | Scheduled Subscriptions in the Music Store Application
 |  | Source Code for This Chapter
 |  | Event State
 |  | The Scheduled Subscription Class
 |  | Testing Scheduled Subscriptions
 |  | Final Testing of the Prototype
 |  | Summary
 | Chapter 7. The SQL-NS Subscription Management API
 |  | Overview of Subscription Management Interfaces
 |  | Types of Subscription Management Interfaces
 |  | The SQL-NS Subscription Management API
 |  | Getting Set Up to Use the API
 |  | Implementing the Logic Layer of a Subscription Management Application
 |  | Building the User Interface
 |  | Summary
 | Chapter 8. Event Providers
 |  | Overview of Event Providers and Event Submission
 |  | Event Provider Concepts
 |  | Preparing to Work with the Source Code in This Chapter
 |  | Using the Built-in Event Providers
 |  | APIs for Building Custom Event Providers
 |  | Building Custom Hosted Event Providers
 |  | Building Standalone Event Providers
 |  | Cleanup: Preparing for the Next Chapter
 |  | Summary
 | Chapter 9. Content Formatters
 |  | Overview of Content Formatting
 |  | Preparing the SQL-NS Instance
 |  | Using the XsltFormatter
 |  | Building a Custom Content Formatter
 |  | Summary
 | Chapter 10. Delivery Protocols
 |  | Overview of Notification Delivery in SQL-NS
 |  | Preparing to Work with Delivery Protocols
 |  | Using the Built-in SMTP Delivery Protocol
 |  | Building a Custom Delivery Protocol
 |  | Restoring Your System
 |  | Summary
 | Chapter 11. Debugging Notification Generation
 |  | Debugging and the Development Process
 |  | Inside the Notification Generator
 |  | Using the SQL-NS Debugging Facilities
 |  | Summary
Part III: Optimization, Deployment, and Administration
 | Chapter 12. Performance Tuning
 |  | Overview of Performance Tuning in SQL-NS Applications
 |  | Streamlining Data Flow
 |  | Optimizing Notification Generation
 |  | Optimizing Notification Distribution
 |  | Summary
 | Chapter 13. Deploying a SQL-NS Instance
 |  | Deployment Configurations
 |  | Configuring a Database System for Deployment
 |  | Vacuuming Old Data
 |  | Deployment Security
 |  | The Deployment Process
 |  | Summary
 | Chapter 14. Administering a SQL-NS Instance
 |  | Monitoring and Reporting
 |  | Maintenance Techniques
 |  | Backup and Restore
 |  | Summary
 | Chapter 15. Troubleshooting SQL-NS Applications
 |  | Using the Troubleshooting Information in This Chapter
 |  | Troubleshooting Compilation
 |  | Troubleshooting Event Submission
 |  | Troubleshooting Notification Generation
 |  | Troubleshooting Notification Distribution
 |  | Summary
Part IV: Advanced Notification Services Concepts
 | Chapter 16. Using Notification Services Management Objects
 |  | Overview of the NMO API
 |  | An NMO Sample Application
 |  | Using NMO to Define and Create an Instance
 |  | Using NMO to Administer an Instance
 |  | Testing the StockBrokerNMO Sample Application
 |  | Using NMO to Reflect the Contents of an Instance
 |  | Summary
 | Chapter 17. Hosting the SQL-NS Execution Engine
 |  | Using the SQL-NS Hosting API
 |  | EngineHost: A Sample Hosting Application
 |  | Summary
 | Chapter 18. User-Defined Matching Logic in SQL-NS Applications
 |  | Overview of User-Defined Matching Logic
 |  | Building the Music Store Application with User-Defined Matching Logic
 |  | Summary
Part V: Appendix
 | Appendix A. Creating a Development Environment for Your Own SQL-NS Applications
 |  | The Structure of the Development Environment
 |  | Extending the Development Environment
Index

Previous Page


Copyright

Microsoft® SQL Server 2005 Notification Services

Copyright © 2006 by Shyam Pather

All rights reserved. No part of this book shall be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein. Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions. Nor is any liability assumed for damages resulting from the use of the information contained herein.

International Standard Book Number: 0-672-32779-1

Library of Congress Catalog Card Number: 2005937218

Printed in the United States of America

First Printing: February 2006

09 08 07 06 4 3 2 1

Trademarks

All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Sams Publishing cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark.

Warning and Disclaimer

Every effort has been made to make this book as complete and as accurate as possible, but no warranty or fitness is implied. The information provided is on an "as is" basis. The author and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book or from the use of the CD or programs accompanying it.

Bulk Sales

Sams Publishing offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact

U.S. Corporate and Government Sales

1-800-382-3419

corpsales@pearsontechgroup.com

For sales outside of the U.S., please contact

International Sales

international@pearsoned.com

Associate Publisher Greg Wiegand

Acquisitions Editor Loretta Yates

Development Editor Songlin Qiu

Managing Editor Charlotte Clapp

Senior Project Editor Matthew Purcell

Copy Editor Barbara Hacha

Indexer Aaron Black

Proofreader Elizabeth Scott

Technical Editor Anand Doshi

Publishing Coordinator Cindy Teeters

Book Designer Gary Adair

Page Layout Nonie Ratcliff

Dedication

This book is dedicated to Phumeza Ethel Manyakanya, for all her love and kindness during her years with my family.


About the Author

Shyam Pather was the Development Lead on the SQL Server 2005 Notification Services team at Microsoft. He joined the Notification Services team when it was formed in late 2000, made key contributions to the product's design and architecture, and implemented several components of the Notification Services compiler and execution engine. Since the first public release of SQL Server Notification Services in August 2002, Shyam has delivered numerous technical talks about notification application development, deployment, and administration at various conferences.

Before joining the SQL Server group, Shyam worked in the Windows Networking organization at Microsoft. As a developer on the Universal Plug and Play (UPnP) team, he helped define the UPnP 1.0 Device Architecture and implemented the UPnP Control Protocol in Windows XP. Prior to working on UPnP, Shyam was a developer in the Windows Network Driver Infrastructure team, working on Raw Channel Access and a unified driver model for voice and data.

After SQL Server 2005 shipped, Shyam moved to Microsoft's Data Programmability team to work on the next generation of data access APIs. He currently runs a small development team building object-relational mapping technology.

Shyam lives in Seattle, Washington. He can be reached via email at spather@microsoft.com or through his website, http://pather.net/shyam.


Acknowledgments

I would like to thank the Notification Services team at Microsoft for building a product I'm proud to write about. The 2005 release is a big milestone for Notification Servicesyour hard work, dedication, and passion made it possible.

Anand Doshi deserves special mention for his work as technical editor of this book. Thank you for the time you put in to reviewing my writing and keeping me honest. Your attention to detail and many helpful suggestions are greatly appreciated.

I'd also like to thank the team at Sams Publishing: Songlin Qiu, Loretta Yates, Kim Spilker, Matt Purcell, and Barbara Hacha. All of you helped me immensely in getting this book together and making it the best it can be. I extend my thanks also to all the people working behind the scenes on the book's production, who I did not get to meet.

To my family and all my wonderful friends, thank you for all your support and for understanding when I couldn't be around. The best part of finishing this book is having more time to spend with all of you.

Finally, my deepest gratitude goes to my girlfriend, Emily. Your love, kindness, and constant encouragement made this book possible. Thank you for always being the perfect companion.


We Want to Hear from You!

As the reader of this book, you are our most important critic and commentator. We value your opinion and want to know what we're doing right, what we could do better, what areas you'd like to see us publish in, and any other words of wisdom you're willing to pass our way.

As an associate publisher for Sams Publishing, I welcome your comments. You can email or write me directly to let me know what you did or didn't like about this bookas well as what we can do to make our books better.

Please note that I cannot help you with technical problems related to the topic of this book. We do have a User Services group, however, where I will forward specific technical questions related to the book.

When you write, please be sure to include this book's title and author as well as your name, email address, and phone number. I will carefully review your comments and share them with the author and editors who worked on the book.

Email: | feedback@samspublishing.com
Mail: | Greg Wiegand
 | Associate Publisher
 | Sams Publishing
 | 800 East 96th Street
 | Indianapolis, IN 46240 USA

For more information about this book or another Sams Publishing title, visit our website at www.samspublishing.com. Type the ISBN (excluding hyphens) or the title of a book in the Search field to find the page you're looking for.


Introduction

A business traveler's cell phone beeps; he's received a text message informing him that his flight's departure gate has been changed. A system administrator receives a message on her pager alerting her that a critical server has gone down. An inventory manager at a grocery store gets an email telling him that the stock level for a popular product has dropped below the threshold for reordering.

These are examples of personalized notifications reaching people when the information is most relevant, on whatever device they have available. In today's information-driven world, notifications such as these have real business value. Timely information can attract customers, make employees more productive, and enable agile decision making. The software systems that generate and deliver these notifications extend the reach of information beyond the corporate desktop.

Building notification software presents several challenges. Every day, people are bombarded with an overload of irrelevant information. For a notification system to add value, it must deliver only targeted, focused information based on the needs and preferences of each user. Doing this correctly requires sophisticated matching and filtering capabilities.

Also, notification systems must be reliable because they're often used to deliver mission-critical alerts. They must be scalable so that they can support millions of users and high volumes of data, without requiring excessive hardware. They must be manageable and fit seamlessly into an existing IT infrastructurethey must not burden the operators who maintain them. Most importantly, notification systems must be secure.

Microsoft SQL Server 2005 Notification Services is a platform for building notification applications that meet all these requirements. It provides a reliable, manageable, secure base for your applications, with a rich, scalable matching engine that can filter an information stream based on the subscriptions of millions of users.

Notification Services offers a simple, declarative programming model that lets you build your application with much less development effort than would be required if you were building it from scratch. Using this programming model, you can get a working prototype of a typical application running in just a single day. Furthermore, the platform offers a number of sophisticated features, such as support for multiple languages and device types, scheduled subscriptions, user-defined matching logic, and multicast delivery, that you can incorporate into your applications by setting a few options. These kinds of beyond-the-obvious features set the most advanced notification applications apart from the rest.

This book is about building notification applications with Microsoft SQL Server 2005 Notification Services. It covers general Notification Services concepts, describes specific coding techniques used to build applications, and walks you through the process of developing a sample application, from the initial design and prototype to final deployment and administration. In addition to the basics, it covers more advanced concepts for developers who want to get the most out of the Notification Services functionality.


Who Should Read This Book

This book is aimed primarily at developers wanting to learn notification application development with Microsoft SQL Server Notification Services. It assumes some familiarity with SQL and XML but no prior background in Notification Services. The book also contains material on deployment, administration, and troubleshooting (Chapter 13, "Deploying a SQL-NS Instance"; Chapter 14, "Administering a SQL-NS Instance"; and Chapter 15, "Troubleshooting SQL-NS Applications") relevant to administrators and IT operators (in addition to developers).

Chapters 7 10 and Chapters 16 18 describe using managed code to build subscription-management interfaces, custom components that extend the Notification Services framework, and applications that embed Notification Services functionality. C# is used in the examples, but all .NET languages are supported. The material in these chapters assumes some familiarity with C#, the .NET Framework, and object-oriented programming.


Teaching Approach

I'm a firm believer that the best way to teach is by example. In this book, I generally explain new concepts in high-level terms first and then go into practical code examples that show those concepts in use. In all cases, the examples form part of a working application you can run.

This book is designed to complement, rather than replace, the Notification Services Books Online (the electronic documentation that comes with the product). Books Online provides a thorough reference to every Notification Services command, programming construct, and configuration option. This book does not attempt to duplicate this reference information. Instead, this book is a guide that shows you how, when, and why to use the various aspects of Notification Services in real scenarios. My expectation is that you will use this book to learn the concepts and work through practical examples, but refer to the Books Online for exact syntax details, usage restrictions, and other detailed reference information.

When providing instructions throughout this book, I've chosen to err on the side of being too specific, rather than not specific enough. In many cases, I explicitly state steps or commands, even though I know most readers will already be familiar with them. My purpose in doing this is to leave no room for ambiguity. I realize that calling out what may seem like obvious instructions can come across as pedantic, but doing this eliminates potential confusion and misinterpretation. I want every reader to get the most out of this book, and to that end I've tried to be as clear as possible at every step. I consider my readers to be capable software engineering professionals, and the tone of the instructions is never intended to contradict that.


Organization of This Book

This book is divided into five parts:

  • Part I, "Notification Services Concepts," (Chapters 1 to 4) provides basic background information on notification applications and the Notification Services platform. It provides instructions for getting your development system set up to work with Notification Services and includes a walkthrough of a sample Notification Services application.

  • Part II, "Notification Services Application Development," (Chapters 5 to 11) covers the specific techniques and tools used to build applications with Notification Services. It describes the development process and the parts of the application that are built at each stage, before taking you through this process with a sample application. In working through the sample application, you learn about the features of the Notification Services platform and how to use them in your own applications.

  • Part III, "Optimization, Deployment, and Administration," (Chapters 12 to 15) covers the steps you perform after the basic development work is completed, to bring a Notification Services application online in a live production environment. It describes performance-tuning tools and techniques, deployment on various hardware configurations, and the administration and maintenance of a running application. It also includes a troubleshooting guide that provides diagnostic information and suggested solutions for the most common problems encountered in running Notification Services applications.

  • Part IV, "Advanced Notification Services Concepts," (Chapters 16 to 18) describes some of the more advanced features of the platform, including many of the new features added in the SQL Server 2005 release. It covers the new programming APIs used to perform Notification Services management tasks programmatically and host the Notification Services engine as a component within another application. It also covers the extensions to the programming model for supporting user-defined matching logic in Notification Services applications.

  • Part V, "Appendix," explains how to set up a development environment for building your own Notification Services applications using the tools and techniques introduced in this book's samples. By following the instructions in the appendix, you can quickly begin developing your own Notification Services applications in a familiar environment.

The first three parts of this book (Chapters 1 to 15) are ordered and organized to match the development process typically followed when creating an application using Notification Services. I expect new readers will generally work through these chapters in order, but this is not a strict requirement. Although each chapter builds on the material in previous chapters, after you have a basic understanding of Notification Services, you should be able to refer to the chapters in any order you choose. The instructions and examples in each chapter are self-contained, allowing you to use them without having to complete all the instructions in previous chapters.

The chapters in the fourth part of the book present three advanced topics. These chapters can be read in any order after you become familiar with the fundamentals of Notification Services application development. These chapters shouldn't be considered required reading unless you specifically want to take advantage of the features they describe. Fully functional Notification Services applications can be built and deployed based only on the material in the first three parts of this book.

Note

You can refer to the book's appendix whenever you're ready to begin creating your own Notification Services applications. If you're unfamiliar with Notification Services, I recommend that you do this only after reading through Parts I and II. The material in the appendix assumes you've become familiar with the development environment used in the samples in Chapters 3 through 10.


About the CD-ROM

The CD-ROM contains the source code and build scripts for all the examples in this book, as well as several tools that aid development and testing. In some places, the chapter text describing the examples shows only code fragments, but all source files on the CD-ROM contain complete code that you can compile and run.

To work with the code and tools on the CD-ROM, you will have to install them on your local hard drive. You cannot work directly off the CD-ROM. Chapter 2, "Getting Set Up," provides instructions for copying the code and tools from the CD-ROM and getting your development system set up to work with them.

Note

Errata information for this book, including updates to the source code, may be periodically posted on the Sams Publishing website. To view this information, visit http://www.samspublishing.com, enter this book's ISBN0672327791 (note that the hyphens are removed)into the Search dialog box, select ISBN as the search term, and click Search to access the book's page.


Conventions

The following typographic conventions are used in this book:

  • Code lines, commands, statements, variables, and any text you see onscreen appears in a monospace typeface. Bold monospace typeface is often used to highlight elements that deserve particular attention in the context of the topic being described. In the case of commands, bold monospace is used to indicate text that you type.

  • In code listings, an ellipsis (...) is used to indicate places where code that isn't shown in the text would belong in a complete source file. This is used in code fragments where parts of the code that are not relevant to the topic being described are omitted for brevity and clarity.

  • Italics highlight technical terms when they're being defined.

  • If a line of code is too long to fit as a single line on the printed page, it is broken into multiple lines. The [image] character appears at the beginning of any printed line of code that is really a continuation of the line immediately preceding it.

  • When not used in the names of XML elements, angle brackets (<>) are used to denote tokens that are substituted with some other value. For example, the name of the Windows service used to run a Notification Services instance is given as NS$<InstanceName>. The <InstanceName> part is replaced with the actual name of the instance. If the instance name was Instance2, the service name would be NS$Instance2.

Caution

Cautions alert you to common pitfalls you should avoid. The cautionary information in this book ranges from general descriptions of restrictions, constraints, or limitations you should be aware of, to more serious warnings about effects that could result in data loss or corruption.

Tip

Tips are used to highlight shortcuts, convenient techniques, or tools that can make a task easier. Tips also sometimes provide recommendations on best practices you should follow.

Note

Notes provide additional background information about a topic being described, beyond what is given in the chapter text. Often, notes are used to provide references to places you can find more information about a particular topic.

Sidebars

Sidebars provide coverage of related topics in a particular area that don't fit within the main organization of a chapter. These topics are usually considered somewhat tangential to the main discussion but provide interesting, useful information nonetheless.


Part I: Notification Services Concepts

IN THIS PART

Chapter 1 An Overview of Notification Applications

Chapter 2 Getting Set Up

Chapter 3 The Simplest Notification Application: Stock Quotes

Chapter 4 Working with SQL-NS Instances


Chapter 1. An Overview of Notification Applications

IN THIS CHAPTER

  • The Publish-Subscribe Phenomenon

  • Components of a Pub-Sub System

  • Challenges in Building Notification Applications

  • SQL Server 2005 Notification Services

This book is about building software applications that connect people to the information they want, when they want it. These applications, called notification applications, are proving to be valuable as businesses seek new ways to reach their customers and employees.

This chapter provides an overview of notification applications and describes some of the technical challenges in building and deploying them successfully. This chapter also introduces Microsoft SQL Server 2005 Notification Services: the development platform that is the focus of this book.


The Publish-Subscribe Phenomenon

Today, most information is accessed via the request-response model illustrated in Figure 1.1: you make a request to a known source, which responds with the information you're looking for. Browsing a website, reading an Internet newsgroup post, or querying customer records in a line-of-business application are all examples of request-response exchanges.

Figure 1.1. The request-response model of information exchange.

The request-response model places the burden on you to request the information you need. Although this is not unreasonable in many circumstances, it is inconvenient when the availability of information is hard to predict. Consider, for example, a website that sells concert tickets. To be sure you don't miss a show by your favorite band when it comes to town, you have to check the website frequently to look at the latest listings. If you forget, or are unable, to check regularly, you could easily miss a show.

An alternative to request-response is the publish-subscribe model, shown in Figure 1.2. In this model, sometimes called the pub-sub model for short, you subscribe for the information you want to receive; whenever that information becomes available, the information source publishes it to you. When you subscribe, the information source maintains a record of what information you want to receive. In pub-sub terminology, this record is called a subscription. A subscription is like a one-time request that may be fulfilled many times in the future.

Figure 1.2. The publish-subscribe model.

Returning to the previous example, the concert ticket website could be extended to support pub-sub, in addition to request-response. The pub-sub component could allow users to subscribe for notices about concerts by their favorite artists. When new concerts are scheduled, it would compare them against the subscriptions and send notification messages to the interested subscribers.

In addition to relieving the burden of requesting information, pub-sub systems can also offer convenience by delivering information over a variety of networks to many different devices. For example, concert notices could be delivered via text-messaging to a subscriber's cell phone. This allows the subscriber to receive the information, even when away from a computer with an Internet connection.

If you're cringing at the thought of information flowing directly to you from publishers, it's because you're thinking of a four-letter word: spam. Whereas spam is unsolicited information, most of which is uninteresting, the information sent by pub-sub systems is usually what the subscribers want to receive because they have knowingly subscribed for it. The goal of pub-sub systems is to provide information that is personalized and timely and reaches users on the devices of their choice.

Pub-sub systems are becoming popular in many industries:

  • Airlines use pub-sub systems to notify their customers of flight delays, cancellations, and gate changes.

  • Stockbrokers use pub-sub systems to notify their clients of significant movement in the stock market.

  • Employment agencies use pub-sub systems to connect employers with potential candidates.

  • Merchants use pub-sub systems to attract customers back when new products become available.

  • Corporations use pub-sub systems to keep employees informed about business activity or operational events.

When it was first introduced, pub-sub functionality was seen as a frill feature: a gimmick tacked onto a few pioneering applications. Now, with the increasing demand for timely, accurate information and more ways to reach people than ever before, pub-sub systems are delivering real business value. This book is about building pub-sub systems with Microsoft SQL Server 2005 Notification Services.

Product Name Clarification

"Microsoft SQL Server 2005" is the full name of Microsoft's relational database product, which includes components such as Notification Services. For brevity, I will drop "Microsoft" from the name in subsequent usage and just refer to the product as "SQL Server 2005." I'll refer to the Notification Services component as "SQL Server 2005 Notification Services" when I need to clearly distinguish the 2005 version from earlier releases; otherwise, I'll just use "SQL Server Notification Services" (abbreviated to "SQL-NS").


Components of a Pub-Sub System

A pub-sub system consists of several components, as shown in Figure 1.3. Several raw information sources feed into the system, providing the information that will later be published to end users. A notification application matches the input information against subscriptions, introduced via subscription systems (usually PCs that connect to the application over a network). As a result of the match, the notification application produces a set of notifications. A delivery system then routes these notifications to the recipients' target devices.

Figure 1.3. Components of a publish-subscribe system.

[View full size image]

The notification application is the core of the pub-sub system. Its job is to match incoming information with subscriptions to produce the notifications to be delivered. SQL Server 2005 Notification Services is a platform for building and running rich, scalable notification applications and integrating them into the framework of a complete pub-sub system. This book teaches you how to leverage this platform to build the notification applications at the center of your pub-sub systems.


Challenges in Building Notification Applications

At first, building a notification application may seem like a trivial task: Gather information, apply some logic to match against subscriptions, and send notifications. When thought about like this, the need for a notification application platform such as SQL Server Notification Services may not be apparent. But, when the system must support thousands, perhaps millions, of subscribers (as commercial systems often must), when it is deployed in mission-critical business environments where reliability is of the utmost concern, and when it is hosted in a data center where security simply cannot be compromised, the seemingly trivial task becomes more challenging. This section explores some of the challenges in building notification applications in more detail. Later in this chapter, the section "SQL Server 2005 Notification Services" (p. 15) explains how these challenges are easily met when using SQL Server 2005 Notification Services as a foundation for your notification applications.

Scalability

The capability of supporting many users and large volumes of data is critical to the success of many notification applications. For example, consider a stockbroker application that notifies subscribers when their stocks cross price thresholds that they specify in their subscriptions. Large volumes of data flow into the application: stock feeds containing many thousands of updated stock prices arrive as often as every few minutes from the stock market. Depending on the size of the broker's client base, there may be tens (or hundreds) of thousands of subscribers, each with several subscriptions for their stock symbols of interest. For such an application to work effectively, it should be capable of handling this kind of volume without excessive hardware.

Even if your intended user base or anticipated data volume is small today, it's important to build your notification application in such a way that it can scale gracefully with growth. After a system is in place and starts being used, it becomes more difficult and more expensive to replace it, so anticipating future growth is vital to keeping IT costs down.

In notification applications, two factors usually affect scalability:

  • The cost (in terms of computing resources) of subscription matching

  • Capacity limitations in the input and output systems

These factors are described in more detail in the following sections.

The Cost of Subscription Matching

Subscriptions can vary in complexity. Revisiting the stockbroker example, the simplest kind of subscription might say, "Notify me when the price of XYZ stock rises above $10." A more complex subscription might say, "Notify me when the price of XYZ stock rises above $10, and the trading volume is greater than 100,000 shares, and XYZ corporation appears in one or more favorable news stories."

In general, more complex subscriptions allow users to be more specific about the notifications they want to receive. But, as subscriptions become richer and more complex, the task of matching them with a set of incoming data becomes more expensive. Consider the computing resources required to match the two subscriptions just described. The first simple subscription is no more than a string match on stock symbol and a simple numeric comparison on stock price. The second, more complex, subscription involves an addition numerical comparison, full-text searching on news stories, and probably some additional metadata querying to determine whether a given news story is favorable.

Obviously, more computing resources are required to evaluate subscriptions of the second kind. This limits the number of those subscriptions that can be supported on a given hardware configuration. Increasing the number of supported subscriptions by limiting their complexity is not really a solution. The richer the subscriptions supported, the better and more interesting the experience for the subscriber. What's needed is a highly scalable matching engine capable of matching complex subscriptions at high enough rates to support the subscriber and data volumes required. Building such an engine is certainly not easy.

Capacity Limitations in the Input and Output Systems

Capacity limitations in the input and output systems are usually outside the control of the notification application developer. Even if you were to build a notification application that could match subscriptions infinitely fast, you'd still be limited by the speed at which the incoming data could be brought in, the capacity of the outbound network from the machine generating the notifications, and the bandwidth limitations of the delivery systems that route the notifications to the end users.

Suppose you built a notification application capable of a subscription-matching rate that produced 500 notifications per second. But, let's say, the network card and protocol implementation on the host system could push out only 100 notifications per second. The easiest way to deal with this (short of getting better network hardware, which is usually expensive) is to split the matching and the delivery tasks and to add additional servers to the delivery end. In other words, one server could generate the 500 notifications per second, and the task of delivering these could be split across five servers, each capable of delivering 100 notifications per second.

This principle applies equally well on the input side. If the amount of incoming data that a single input machine can supply is limited, you can split the task of submitting the input data across several machines to increase the overall capacity of your system. In general, adding machines to increase scale is referred to as scaling out.

Scaling out requires the application to be architected to support a distributed deployment. Getting this right is often a substantial development task.

Reliability

Many notification applications are customer facing, and so they often come associated with certain quality-of-service requirements. Users simply expect to get their notifications, and if they don't, they quickly become dissatisfied.

To meet these customer expectations, a notification application must seldom (hopefully, never) crash, restart gracefully after downtime (planned or unplanned), and work with high-availability clustering systems.

In addition, notification applications must be capable of dealing with the reality of failures in the infrastructure around them. Incoming data feeds don't always arrive as expected or may contain erroneous information. Delivery systems sometimes fail. Notification applications should be capable of detecting (possibly even correcting) problems in incoming data feeds or alerting an administrator if feeds don't arrive as expected. Should notifications fail to be delivered, the application must be capable of retrying the delivery at a later time, using some intelligence to handle prolonged network downtime as well as random, intermittent failures.

Rich Application Features

When first thinking about building a notification application, developers usually think in terms of what the incoming data looks like, what kinds of subscriptions to support, what kind of logic it takes to match the two, and what the final notifications look like. Although these aspects are at the core of the application, they are usually not the parts that take the most time and effort to build, test, and debug.

When building a notification application without the help of a platform framework such as SQL Server Notification Services, the bulk of the development time is spent building features such as digesting (aggregating several notifications for the same subscriber into a single message), multicasting (sending the same notification to several subscribers), support for multiple languages, and scheduled subscriptions (subscriptions that fire at a specific time of day, rather than in response to any particular incoming information feed). Also, features that make the application manageable, such as diagnostic capabilities, reporting tools, and administration facilities, are often time-consuming to build.

Security and Privacy

Notification applications usually maintain personal data in the form of subscriptions that describe what information each user wants to receive, as well as the delivery addresses of the devices on which they receive it. Also, they have inbound and outbound connections that can be potentially dangerous if compromised. Therefore, security should be the first concern of anyone building a notification application.

Subscriber and subscription data must be kept secure and private. No unauthorized person should be able to read or modify this data. Furthermore, subscriptions should be isolated: No subscription should affect the results of any other.

Any inbound or outbound network connection should be treated as a potential security vulnerability. Regardless of what security measures are in place, a notification application must be designed for defense in depth, so that if a network connection were compromised and used by an attacker to gain entry into the system, his ability to do damage would be limited. A notification application should be designed to run under a low-privileged account so that it doesn't become a target for code-injection attacks.

Because notification applications, by nature, send messages, if compromised, they can be used to spam users or launch denial of service attacks. Auditing and throttling mechanisms need to be in place to prevent such malicious misuses should a security breach occur.

In addition, notification applications may be connected to one or more database servers on which data owned by other systems and applications is stored. Notification applications need to be designed so that no attacker could use them to gain unauthorized access to other data on the database servers.


SQL Server 2005 Notification Services

SQL Server 2005 Notification Services (SQL-NS) is a platform for building and executing rich, scalable notification applications. It's part of the Business Intelligence Suite in SQL Server 2005.

Note

The previous version of SQL Server Notification Services (version 2.0) was released as a web download addition to Microsoft SQL Server 2000. The information in this book applies only to the 2005 release. For information on version 2.0, see the previous edition of this book, Microsoft SQL Server 2000 Notification Services, ISBN: 0672326647.

A Platform for Building Notification Applications

If you looked at the internals of several notification applications, you would see certain patterns in common. At the highest level, they all gather data, match it with subscriptions, and generate notifications. Digging deeper, you'd notice that the software components to accomplish these tasks all shared a basic framework that was customized to the specifics of each application. If you were building several such notification applications, you wouldn't rebuild this common framework for each one; instead, you'd abstract the common elements in a set of components and then add application-specific code where necessary.

SQL-NS is Microsoft's implementation of the software elements common to all notification applications. It is a platform on which rich, scalable, reliable notification applications can be built much faster than if they were built from scratch. Using this platform, it is possible to prototype a working notification application in a single day of development. Many customer-facing notification applications built on SQL-NS have gone from design to running live in production within eight weeks. Many applications designed for internal use have been deployed even more quickly.

SQL-NS provides a declarative programming framework. You specify the size and shape of the data (incoming data feeds, subscription data, and notification data) and the logic used to determine when a piece of incoming data matches a subscription. The framework provides the mechanisms for facilitating the information flow, executing the logic, and performing the notification delivery. The result is that you can focus on the specifics of your application and not worry about the platform fundamentals.

The programming framework offers easy-to-use abstractions of the rich application features, such as digesting, retry on failure, and support for scheduled subscriptions discussed in the section "Rich Application Features" (p. 14). Applications built on SQL-NS have all these features, without your having to implement them yourself. Features like these distinguish the most sophisticated notification applications from the rudimentary ones that don't usually generate customer satisfaction.

Frameworks that support a wide array of applications always impose some restrictions on the developers who build on them. Although this is certainly true of SQL-NS, my experience working with a number of developers who have used it in the real world has shown that the vast majority of notification applications can be built on SQL-NS quite naturally. And the framework is extensible. In cases where it does not fit the needs of your application, you can customize it. Plug-in components (that you implement in any of the .NET programming languages) can be inserted at many points in the framework to enhance or replace existing behavior.

A Platform for Executing Notification Applications

In addition to the declarative programming framework, SQL-NS also provides an engine for executing the notification applications you build. This engine is based on SQL Server 2005 and the Microsoft .NET Framework 2.0. It provides the scalability, reliability, and security required by commercial notification applications. By running your application on the SQL-NS engine, you get these benefits automatically. Throughout this book, we will develop applications using the SQL-NS development platform and then run them using the SQL-NS engine.

The SQL-NS engine is designed to be easy to deploy and administer. It supports a multitude of deployment scenarios, including single-server, distributed, and clustered configurations. It comes with a complete set of management tools in SQL Server 2005 Management Studio. It also provides monitoring, reporting, and diagnostic facilities that integrate with existing infrastructure in IT environments.

Why Use SQL Server Notification Services?

SQL-NS is proven technology being used in live production applications by several large companies, including Microsoft itself.

Though it may seem so at first, building a professional notification application is not a trivial task. The key benefits of using SQL-NS to build your application can be summarized as follows:

  • Developer productivity SQL-NS dramatically reduces the development effort required to build your applications and give them the performance, reliability, security, and rich features needed.

  • Administrator convenience Applications built on SQL-NS behave predictably, can be deployed using proven practices, and are supported using a standard set of tools and procedures.

Combined, these factors greatly reduce the time to market for most notification applications. After a SQL-NS-based notification application is up and running, these same factors contribute to keeping its maintenance costs low.

If you already have SQL Server 2005, you can begin using SQL-NS immediately. Chapter 2, "Getting Set Up," provides the instructions on how to install SQL-NS and set up your environment to use it.

Tip

If you are thinking about building a notification application, I strongly encourage you to try SQL-NS and work through some of the samples in this book. Doing so will give you some first-hand insight into what SQL-NS offers and will likely help you get your application up and running quickly.

What's New in 2005

For readers familiar with previous versions of SQL-NS, this section highlights the new features and modifications in the 2005 release. Although the basic application model has remained the same, the following enhancements make the product richer, more flexible, and easier to use:

  • Integration with SQL Server 2005 SQL-NS is now packaged with SQL Server 2005, rather than as a separate download. It is installed via the SQL Server 2005 installer and integrated into the SQL Server 2005 management and upgrade tools. Because the new version of SQL-NS is based on SQL Server 2005, notification applications can take advantage of new database engine features, such as native XML support and hosting of CLR assemblies.

  • 64-bit support SQL-NS now runs on the 64-bit platforms supported by SQL Server 2005, in both native and Windows-on-Windows (WoW) modes.

  • End-user defined matching logic The enhanced matching engine in SQL-NS 2005 enables you to build applications in which the subscribers themselves can provide the matching logic for their subscriptions. In previous releases, the matching logic was always supplied by the application developer. The new behavior enables you to build more powerful applications that users can customize freely.

  • Analysis Services event provider SQL-NS 2005 includes a new, built-in event provider that executes MDX queries against cubes built with Microsoft SQL Server Analysis Services. Using this component, you can build applications that obtain input data from multidimensional sources, without writing custom code.

  • Hostable execution engine The SQL-NS execution engine can now be hosted inside any application process. Previously, the SQL-NS execution engine had to run as a separate Windows service. Using this feature, you can embed SQL-NS functionality into an existing application.

  • Database independence The database objects (such as tables, views, and stored procedures) that SQL-NS installs when creating notification applications can now be deployed into any database. Previously, SQL-NS database objects had to be deployed to dedicated SQL-NS databases, which sometimes created inconvenience for database administrators.

  • Management API All SQL-NS management operations, including creating new SQL-NS applications, can be done programmatically.

  • Improved testing and debugging facilities With the 2005 release, testing and debugging SQL-NS applications is easier than ever. It is now possible to test and debug an application without writing any custom code to enter input data.

The sample code throughout this book illustrates these new features in use.


Summary

In today's business climate, timely information can attract customers, increase employee productivity, and enable agile decision making. In cases where the availability of information is unpredictable, consumers, information workers, and businesses are increasingly choosing to have information pushed to them directly from trusted sources, rather than having to find it themselves. They want information to reach them when it's most useful, wherever they might be, on whatever device they may have with them.

Enabling this kind of rich, personalized information delivery requires a publish-subscribe system. The key component of any publish-subscribe system is the notification application that gathers incoming information, determines which subscribers want to see it, and generates notifications for those subscribers.

Building a notification application from scratch is a task riddled with difficulties. Scalability, reliability, security, and privacy present big challenges, and building the rich features required to truly satisfy customers can consume the bulk of the development schedule.

SQL-NS is a platform for building and executing rich, scalable notification applications. It can greatly reduce the time to market for your notification applications because it does much of the difficult work for you. SQL-NS is an integrated part of SQL Server 2005. If you're building a notification application, you should invest the time to learn SQL-NS.


Chapter 2. Getting Set Up

IN THIS CHAPTER

  • Your Development Environment: The Big Picture

  • Prerequisites and System Requirements

  • Standard or Enterprise Edition?

  • Setting Up Your Development Environment

This chapter explains how to get your system ready to develop applications with SQL-NS. We'll cover how to install SQL-NS and set up your development environment to build and run the sample code in this book.

Caution

This is a one-time setup that will be used in all subsequent chapters of this book. It's important that you take the time now to get properly set up before moving on.

SQL-NS is a flexible product and can be installed in a number of ways, depending on your particular environment and needs. The way you install and configure SQL-NS on the systems you use to develop your applications will likely differ from the way you install and configure it on the production systems on which you deploy your finished applications. This chapter describes the most common installation scenarios for development. Chapter 13, "Deploying a SQL-NS Instance,"addresses deployment and describes how to install SQL-NS in several types of production environments. In addition to working with this book's sample code, you'll be able to use the development environment set up in this chapter to develop your own SQL-NS applications as well.


Your Development Environment: The Big Picture

Before going into the specific hardware and software requirements for SQL-NS, it's important to understand the high-level picture of the development environment you'll need to set up. This section describes the environment in general terms, and the following sections provide detailed instructions on how to set it up.

Terminology Clarification

The term "SQL Server" has several meanings, depending on the context in which it is used. When used as a noun by itself, such as "connect to your SQL Server," it generally refers to an instance of the relational database server. This is also sometimes referred to as the "database engine." When used as a qualifier to another noun, such as "consult the SQL Server documentation," it usually refers to the SQL Server 2005 product as a whole. Throughout this chapter and the remainder of this book, I have tried to use these forms consistently.

Computers

First, you'll need a computer on which to do your development. You'll install the SQL-NS tools and execution engine on this computer. You'll use the SQL-NS tools to write, compile, and debug your applications and you'll invoke the SQL-NS execution engine to run them.

You'll also need a SQL Server 2005 database server. When you build your applications, SQL-NS creates or modifies databases on this SQL Server, and when your applications run, they manipulate the data in these databases.

The simplest and most convenient arrangement is to have the SQL Server run on your development computer. Throughout the remainder of this book, I'll refer to this configuration as the local SQL Server configuration. If you are able to set up your system this way, I highly recommend doing so. (Remember that the recommendations in this chapter are for development only; in deployment, it may be best to run your SQL Server on a different computer than the SQL-NS engine.)

If the constraints in your environment prevent you from installing a local SQL Server on your development computer, don't be concerned; SQL-NS fully supports development against a SQL Server that runs on a remote computer. This chapter includes instructions on how to set up your environment to support this. I'll refer to this as the remote SQL Server configuration.

In the following sections (and in the rest of this book), I'll refer to the computer on which you do your development as your development machine. I'll refer to the computer running the SQL Server database engine as your SQL Server computer, or just your SQL Server. If you're using the local SQL Server configuration, the development machine and the SQL Server will be the same physical computer.

Tools for the Development Process

The development process of a SQL-NS application is similar to that of any other application: You write code, compile that code, and then test and debug it. The following sections describe the tools you'll need to perform these steps in the development process.

SQL Server 2005 Management Studio

SQL Server 2005 Management Studio is the new, standard tool for managing SQL Server 2005 databases. It's a part of the client tools suite that ships with SQL Server 2005. In addition to supporting the usual database management operations, Management Studio also provides powerful T-SQL and XML editors, as well as other tools used in database application development. If you're familiar with the SQL Server 2000 client tools, think of Management Studio as a combination of Query Analyzer and Enterprise Manager, integrated with some features of the Visual Studio .NET development environment.

Building a SQL-NS application mainly involves writing XML documents that declare the application's data types, logic, and configuration, processing those XML documents with the SQL-NS compiler, and testing the application using T-SQL scripts. Management Studio is an ideal tool for doing all these tasks. The XML editor built in to Management Studio provides syntax highlighting, syntax checking, XSD validation, and IntelliSense statement completion, all of which make the task of defining SQL-NS applications easier. The T-SQL editor is just as powerful and useful when testing SQL-NS applications. In addition, all the SQL-NS compilation and management tools are available directly in the Management Studio user interface.

In all the examples in this book, Management Studio will be the primary development environment for building SQL-NS applications. Chapter 3, "The Simplest Notification Application: Stock Quotes," introduces the first sample SQL-NS application and uses Management Studio to examine its XML definition. Chapter 4, "Working with SQL-NS Instances," explains the SQL-NS tools built in to Management Studio in detail.

Visual Studio .NET 2005

Visual Studio .NET 2005 is Microsoft's premier development environment for building applications based on the .NET Framework 2.0. Some aspects of building a SQL-NS application involve writing procedural code, and Visual Studio .NET 2005 is well suited to these tasks.

For example, if you're building a user interface for subscription management or a custom plug-in component to extend the SQL-NS framework, you'll probably be writing managed code (in C#, VB .NET, or any of the other .NET languages). Chapters 7 through 10 cover these aspects of SQL-NS application development and explain the use of Visual Studio .NET 2005.

Although Visual Studio .NET 2005 certainly makes managed code development easier, it is not strictly required to build a SQL-NS application. You can also build all the managed code components using just the .NET Framework 2.0 SDK (available as a free download from Microsoft's MSDN website, http://msdn.microsoft.com). However, all the managed code examples in this book assume you are using Visual Studio .NET 2005.

Caution

Because all components of a SQL-NS 2005 application must be based on version 2.0 of the .NET Framework, you cannot use earlier versions of Visual Studio .NET for SQL-NS 2005 development. Visual Studio .NET 2005 is the first version to support the .NET Framework 2.0. Visual Studio .NET 2003, for example, supports only the .NET Framework 1.1.

Command-Line Tools

The SQL-NS compiler and all the other SQL-NS tools are available in command-line versions, in addition to the graphical versions built in to SQL Server 2005 Management Studio. The command-line versions are useful for automating SQL-NS management tasks through scripts. In my experience, most SQL-NS users find the command-line tools most convenient to use after they have learned the basics of SQL-NS, but the graphical versions are more user friendly the first time around.

In Chapter 3's walk-through of the first SQL-NS application in this book, the instructions refer to the graphical tools only. In Chapter 4, I explain both sets of tools in detail. In subsequent chapters, instructions reference the command-line tools only, but you should have enough information to perform the same steps using the graphical tools if you prefer.

Accounts and Security

It used to be commonplace, not so long ago, to run everything as Administrator and turn off all security features. Things were easy to configure and deploy, and you would never see those annoying Permission Denied error messages.

But hackers and virus builders exploited these careless practices. Nowadays, you have to be more careful. In every topic covered in this book, we're going to pay attention to security and configure everything we build as securely as possible. This is going to mean a few extra steps to get things running and will require some thinking about user accounts and their permissions up front. But as you will see, maintaining a focus on security is not difficult and gives you tremendous peace of mind.

SQL-NS is designed to enable notification applications to run securely. Although SQL-NS development and administration tasks require Administrator permissions, the SQL-NS runtime execution components are designed to run in much lower-privileged security contexts.

When thinking about security for SQL-NS applications, the first account to consider is the one you use to log in to your development machine to build your applications. This account requires administrative privileges on the development machine because it'll be installing Windows Services and creating Registry keys, among other operations.

The account will also need system administrator permissions on the target SQL Server because some of the operations you do during SQL-NS application development install and manipulate tables in system databases. The section "SQL Server Authentication Modes" (p. 25) describes the various ways you can connect to your SQL Server; the particular mode you choose affects the way you configure your SQL Server permissions. Specific instructions for setting up SQL Server permissions are provided in the section "Setting Up SQL Permissions and Accounts" (p. 32).

The other account to consider is that under which the SQL-NS engine runs. The engine is usually run as a Windows Service, which you configure to run under a particular account. It's recommended that this account not have Administrator privileges. That way, in the unlikely event that the service should become compromised by a hacker or a virus, any code injected into the service process will not be capable of doing much harm. The account under which the engine runs needs only a few specific permissions on the SQL Server to run your applications. You'll grant the service account basic login access to the SQL Server as you work through the instructions in this chapter; you'll grant the other permissions it requires when you create each application. (The required permissions are explained in Chapter 4.)

Note

Chapter 17, "Hosting the SQL-NS Execution Engine," describes how you can host the SQL-NS engine in any application process. When hosted this way, the SQL-NS engine will assume the credentials of the hosting process. The account under which that process runs will need to be granted the same permissions normally granted to the Windows Service account.

SQL Server Authentication Modes

SQL Server offers two authentication modes by which a user can log in to the database server: Windows Authentication and SQL Server Authentication. With Windows Authentication, Windows credentials are used to authenticate the user and log in to the database server. With SQL Server Authentication, at the time of the connection the user supplies a SQL Server username and password, which are used to authenticate the user and log in. The SQL Server username and password used in SQL Server Authentication may be different from the Windows username and password of the user making the connection.

Windows Authentication is recommended because it's the more secure of the two methods and is easier to configure, but both methods are supported. As described in the previous section, both the account under which you do development and the account under which the SQL-NS engine runs need to connect and log in to the SQL Server. You can use either Windows Authentication or SQL Server Authentication for these connections.

Your choice of authentication mode affects how the SQL-NS command-line tools and the SQL-NS engine are configured. The "Environment Variables" section (p. 36) describes the settings that control the authentication mode used by the application examples in this book. Chapter 4 describes other configuration steps required for each of the modes.


Prerequisites and System Requirements

To install and use SQL-NS, your system must meet the minimum hardware and software requirements for SQL Server 2005. The details of these requirements are documented on the SQL Server 2005 website at http://www.microsoft.com/sql/prodinfo/sysreqs/default.mspx.

The website is updated as new service packs and updates are released.

The SQL-NS Website

Microsoft's official SQL-NS website is http://www.microsoft.com/sql/ns

This website provides links to an online version of the product documentation, licensing information, whitepapers, and other technical resources. It's also a good place to look for news about SQL-NS, including product updates.

Throughout this chapter, I will refer to URLs beneath this main SQL-NS web page. Please be aware that these URLs might change as Microsoft revises the layout of the website. If, at the time you read this, a particular URL I've mentioned does not appear to work, start from the root of the website (the preceding URL is not likely to change) and try to find the information by navigating the links provided or by using the search facility.


Standard or Enterprise Edition?

SQL-NS is available in four editions of SQL Server 2005: Developer Edition, Evaluation Edition, Standard Edition, and Enterprise Edition. The functionality of Developer Edition and Evaluation Edition is the same as Enterprise Edition; the difference is that Developer Edition is licensed only for development use and Evaluation Edition is licensed only for evaluation purposesneither can be put into production. Therefore, when comparing the editions, we usually distinguish only the Standard and Enterprise Editions in terms of SQL-NS features. Developer and Evaluation Editions can be substituted for Enterprise Edition in development and evaluation environments.

Caution

SQL-NS is not supported on the Express or Workgroup editions of SQL Server 2005.

Although the core SQL-NS functionality in the two editions is similar, the Enterprise Edition supports several features that make it more scalable and therefore capable of supporting the largest applications. Specifically, Enterprise Edition supports the following features, which Standard Edition does not:

  • Distributed deployment With Enterprise Edition you can run different parts of your SQL-NS applications on different physical computers, enabling you to scale out your system.

  • Thread pool and notification batch size settings Enterprise Edition offers options that let you fine-tune resource allocation so that you can precisely control how the workload of your SQL-NS application is handled.

  • Multicast delivery For applications built on Enterprise Edition, you can choose to distribute the same physical notification to multiple subscribers, resulting in better performance. In Standard Edition, you have to distribute a separate notification for each subscriber.

It should be noted that even in the Standard Edition, SQL-NS does support very large applications because of the inherent scalability of the application model and execution engine. The Enterprise Edition features take scalability even further.

Because the SQL-NS components may be installed independently of the SQL Server used to host SQL-NS applications (especially in remote SQL Server configurations), it's possible for their respective editions to differ. For example, the SQL-NS tools and engine may be Enterprise Edition (because they were installed from a SQL Server 2005 Enterprise Edition CD), but the SQL Server itself may be Standard Edition. The following are the licensing restrictions on edition mismatches:

  • Only Standard Edition or Enterprise Edition SQL Servers may be used to host databases for SQL-NS applications.

  • The edition of the SQL-NS tools need not match the edition of the SQL Server against which they operate. For example, Enterprise Edition SQL-NS tools may be used to create a SQL-NS application on a Standard Edition SQL Server and vice versa.

  • The edition of the SQL-NS engine used to run a SQL-NS application must be the same as the edition of the tools used to create the application.

The edition of the SQL-NS tools used to create an application determine the functionality that may be used in that application. For example, an application that uses multicast delivery, an Enterprise Edition feature, can only be built using the Enterprise Edition SQL-NS tools. Only an Enterprise Edition SQL-NS engine may run such an application.

You can use the SQL-NS tools and engine from either edition to work through the samples in this book. Subsequent chapters cover features that are available only in the Enterprise Edition (they are all marked as such), but those features are not used in the samples.


Setting Up Your Development Environment

This section describes the steps you'll need to complete to get set up to work with SQL-NS and this book's sample code. You'll begin by installing SQL-NS and all required software, and then you'll copy and configure the sample code from this book's CD-ROM.

Installing Notification Services and Other SQL Server 2005 Components

SQL Server 2005 comes with one setup program from which you can install all components. This single setup program lets you choose to install the database engine, Analysis Services, Reporting Services, Notification Services, Integration Services, or the client tools and documentation. You can select various combinations of these components to install on any particular computer. Follow these steps on your development machine to install the components required for SQL-NS development:

1. | Insert your SQL Server 2005 DVD and launch setup (or launch setup from a network location, if appropriate).
2. | Setup will check for any missing software prerequisites and install them if necessary.
3. | The setup wizard begins. The first step performs a configuration check to verify that your system is ready to install SQL Server 2005. The details grid in the wizard dialog box reports the status of each item examined. If you see error or warning icons next to any items in the list, you may have to perform additional steps to prepare your system for SQL Server 2005 installation. Click the Report button and select View Report from the drop-down menu to see a detailed explanation of the checks that were performed. The report may suggest remedies to any problems encountered.
4. | If the system configuration check found issues that prevent setup from continuing, you will have to exit the setup wizard, fix the problems as suggested, and then launch setup again. If no blocking problems were found, click the Next button to proceed to the next step in the wizard.
5. | In the registration dialog box, enter your registration information, including your product key.
6. | The next step in the setup wizard presents you with a choice of components to install. To install the SQL Server 2005 database engine on your development machine, select the SQL Server Database Services option. If you have already installed the database engine on your development machine, or if you plan to develop your SQL-NS applications against another existing database engine instance, you do not need to select the SQL Server Database Services option.
7. | Select the Notification Services option to install the SQL-NS components.
8. | Select the Workstation Components, Books Online and Development Tools option to install the SQL Server tools and documentation, including Management Studio, if you do not already have these installed on your development machine.
9. | Complete the rest of the setup wizard to finish the installation.

After you successfully complete the setup wizard, you'll see a new group called Microsoft SQL Server 2005 added to your Start menu under Programs. This program group contains shortcuts to all the tools we'll use to develop SQL-NS applications, as well as the SQL Server Books Online (detailed product documentation for all aspects of SQL Server 2005, including SQL-NS). From now on, I'll refer to this program group as the SQL Server program group.

Installing Visual Studio .NET 2005

If you do not already have Visual Studio .NET 2005 installed on your development computer, it's a good idea to install it now. To work with the managed code examples in this book, the Visual C# language and tools must be in the list of Visual Studio components you choose to install.

Note

Because Visual Studio .NET 2005 comes in many editions, each with its own set of prerequisites, dependencies, and components, it's not practical for me to provide detailed instructions on installing Visual Studio .NET 2005 here. Refer to the Visual Studio .NET 2005 documentation for thorough information on all the installation options.

Installing the Source Code

All the source code used in this book is located on the accompanying CD-ROM.

To work with it, you have to copy it to the hard drive of your development machine; you cannot use it directly from the CD-ROM. The following steps describe how to properly copy the source code:

1.
Insert the CD-ROM into the CD-ROM drive of your development machine.

2.
Open a command prompt. Enter the commands described in the following steps in this command prompt.

3.
Change to the drive on which you want to put the source code. On my system, this is the C: drive, but you can use any drive on your system you choose. (Replace "C:" in the following commands with the appropriate drive letter.):

C:

4.
Create a directory in which to place the source code. On my system, the directory I use is C:\SQL-NS, but you can call it whatever you want and put it anywhere in the filesystem. If you choose a different location, replace C:\SQL-NS in all the following commands with the directory path you've chosen. If your directory name contains spaces, be sure to enclose it in quotes in all the following commands.

mkdir C:\SQL-NS

5.
Copy the source code from the CD-ROM. My CD-ROM drive is assigned the drive letter E:. Your CD-ROM drive may have a different drive letter, so replace E: in the following command with the appropriate drive letter:

xcopy /ev E:\SourceCode\*.* C:\SQL-NS

This copies all the required files to your development machine. If you explore the folder you copied the source code to, you should see the following subdirectories:

  • Chapters Contains source code and other files specific to each chapter in this book.

  • Common Contains configuration scripts and other files shared among all the examples in this book.

  • Samples Contains several working SQL-NS sample applications on which this book's examples are based.

  • Tools Contains utilities used in developing and testing SQL-NS applications.

As you work through the remaining chapters of this book, you'll become familiar with the meaning and use of the files in these directories.

Make a note of the directory you created in step 4 in the preceding list of instructions. You'll need to refer to this directory several times as you work through the samples in this book. From now on, I'll refer to this directory as the source code base directory.

Creating the Service Account

In most of this book's samples, we'll run the SQL-NS engine as a Windows Service. You must configure the service to run under a particular account. For security reasons, it's recommended that you run the service under a low-privileged (non-Administrator) account. This section describes how to create an account you can use to run the SQL-NS engine.

The account you create can be either

  • A local account

  • A domain account

A local account exists only on the machine on which you create it and generally cannot access resources on other machines. Domain accounts are created on the domain controller and are recognized by all the machines in the domain. Which type of account is appropriate in your environment depends on the answers to the following questions:

  • Is your development machine joined to a domain?

  • Is your SQL Server local (on the development machine) or remote (on another computer)?

  • Do you want to use Windows Authentication or SQL Server Authentication with your SQL Server?

If your development machine is not joined to a domain, you must create a local account, and you can use Windows Authentication only if your SQL Server is local. If your development machine is not joined to a domain and your SQL Server is remote, you must use SQL Server Authentication.

If your development machine is joined to a domain, you can create either a local or a domain account, but this choice affects the authentication modes available to you if your SQL Server is remote:

  • If you create a local account, you must use SQL Server Authentication.

  • If you create a domain account, you can use either Windows Authentication or SQL Server Authentication.

If your development machine is joined to a domain and your SQL Server is local, you can use either authentication mode regardless of which type of account you create.

All the preceding choices are summarized in Figure 2.1.

Figure 2.1. Matrix of choices for account types and authentication modes.

[View full size image]

This section provides separate sets of instructions for creating a local account and creating a domain account. Based on the information in Figure 2.1, decide whether you'll be creating a local account or a domain account and follow the appropriate set of instructions. Before doing this, however, you must first decide on a username and password for the account. I have chosen to give the account I use to run the SQL-NS service the name SQL-NS_Service. You can choose any other name if you prefer; if you choose a different name, replace SQL-NS_Service in all the following commands with the name you've chosen.

Caution

Make sure you remember the username and password you choose for the SQL-NS service. You will need them throughout your work with the code samples in this book.

To create a local account, follow these steps:

1.
Open a Command Prompt on your development machine and issue the following command (be careful not to omit the * after the username):

net user "SQL-NS_Service" * /add

2.
When prompted, enter the password you've chosen for the account. After entering the password, you are asked to confirm it.

To create a domain account, follow these steps:

Note

Depending on how your domain is set up and administered, you may not have permission to create a domain account. If this is the case, your domain administrator must create the account for you.

1.
Open a command prompt on your development machine and issue the following command (make sure that you don't omit the * after the username):

net user "SQL-NS_Service" * /add /DOMAIN

2.
When prompted, enter the password you've chosen for the account. After entering the password, you are asked to confirm it.

Setting Up SQL Permissions and Accounts

You need to perform a few steps to get the accounts and permissions on your SQL Server set up appropriately for SQL-NS development. These steps differ based on whether you're using Windows Authentication or SQL Server Authentication. The following sections describe the specific instructions for each authentication mode.

Windows Authentication

Windows Authentication relies on Windows credentials to authenticate users when they attempt to log in to the database server. If you're using Windows Authentication, configuring your system for SQL-NS development involves granting specific forms of SQL Server access to various Windows user accounts.

As described in the "Accounts and Security" section (p. 24), both the SQL-NS development tools and the SQL-NS runtime engine make connections to the database server. The tools are usually invoked in the context of the Windows user account under which you're logged in to your development machine. In contrast, the SQL-NS engine usually runs as a Windows service, in the context of the service account you set up in the "Creating the Service Account" section (p. 30). You'll need to configure your SQL Server to allow both the Windows user account you use on your development machine, and the service user account, to log in. However, the two accounts will be granted different levels of privilege.

By completing the following instructions, you'll enable your development account and the service account to log in to the SQL server. You'll also add the development account to the sysadmin fixed server role, making it a system administrator on the database server. This is necessary because some of the development tools manipulate system tables. The service account does not require system administrator permissions; other permissions needed to perform various operations in the databases associated with each application will be granted to the service account later.

Caution

If you're using SQL Server Authentication, do not perform these steps: You can skip this section entirely and move instead to the "SQL Server Authentication" section that follows (p. 34).

Note

The following instructions must be performed by a user who is already a system administrator on the SQL Server.

1.
Open Management Studio (from your Start menu, choose Programs, Microsoft SQL Server 2005, SQL Server Management Studio).

2.
In the Connect to Server dialog box, choose Database Engine as the server type, specify the name of your SQL Server instance in the Server name text box, and choose Windows Authentication in the Authentication drop-down list. Click the Connect button to connect to the SQL Server.

3.
Click the New Query button in the toolbar to open a new query window.

4.
Enter the following SQL statements:

USE master

IF NOT EXISTS (
    SELECT name
    FROM sys.syslogins
    WHERE name = 'DOMAIN\user'
)
    CREATE LOGIN [DOMAIN\user] FROM WINDOWS
GO
EXEC sp_addsrvrolemember 'DOMAIN\user', sysadmin

5.
Replace DOMAIN\user with the domain name and username of your development account (the Windows account under which you log in to do SQL-NS development). You'll need to make three replacements. Note that if your development account is a local account, DOMAIN should be replaced with the name of your development machine.

6.
Execute the query by pressing F5.

7.
Clear the query text (or open a new, blank query) and enter the following SQL statements:

USE master

IF NOT EXISTS (
    SELECT name
    FROM sys.syslogins
    WHERE name = 'DOMAIN\user'
)
    CREATE LOGIN [DOMAIN\user] FROM WINDOWS

8.
This time, replace DOMAIN\user with the domain name and username of your service account (the Windows account you created in the "Creating the Service Account" section, p. 30). You'll need to make only two replacements. If your service account is a local account, DOMAIN should be replaced with the name of your development machine.

Caution

Notice that the statements in step 7 do not include the call to sp_addsrvrolemember that was made in step 4 because the service account does not need to be a member of the sysadmin role.

9.
Execute the query by pressing F5.

SQL Server Authentication

This section describes how to create and configure the SQL accounts you'll need if you're using SQL Server Authentication.

Caution

If you're using Windows Authentication, do not perform these steps: You can skip this section entirely. You should, however, have performed the instructions in the section "Windows Authentication" (p. 32).

When building SQL-NS applications, you need two SQL Server accounts:

  • One to use when doing development and administration work

  • One to use when testing your applications

Note

As with everything else in this chapter, the account configuration described here applies only to development. When you deploy your SQL-NS applications in production, you may need different SQL Server accounts, and they may be configured with different security settings.

The first account is the one that the SQL-NS tools use to connect to the SQL Server when you compile and configure your applications. I'll refer to this as the development account. When you invoke the SQL-NS tools, you specify the username and password of this account. This account must have system administrator permissions on the SQL Server to perform the operations required to create and manage SQL-NS applications.

The second account is used at runtime by the SQL-NS engine and other application components when they need to read and manipulate your application's data. I'll refer to this as the test account. When you configure the SQL-NS engine, you supply the username and password of this account. The test account requires specific permissions in the SQL-NS databases it accesses, but it does not need to be a system administrator.

You can use the same SQL Server account for both development and testing, but I don't recommend this. It's generally good security practice to always connect to your SQL Server with the minimum required permissions for the operations you're performing. If you used a single account for development and testing, you'd always be connecting as a system administrator. Though this is necessary when creating and managing SQL-NS applications, the runtime components of SQL-NS don't need this level of privilege.

I recommend that you use two accounts and configure each of them as described in the following sections.

The Development Account

Choose a username and password for your development account. On my system, I've chosen the username SQL-NS_Dev, but you may use another username if you prefer. Be sure to remember the username and password you've chosen; you'll need them later.

Note

The following instructions must be performed by a user who is already a system administrator on the SQL Server.

1.
Open Management Studio (from your Start menu, choose Programs, Microsoft SQL Server 2005, SQL Server Management Studio).

2.
In the Connect to Server dialog box, choose Database Engine as the server type, specify the name of your SQL Server instance in the Server name text box, and choose SQL Server Authentication in the Authentication drop-down list. Enter the SQL username and password of a system administrator account in the User name and Password text boxes. Click the Connect button to connect to the SQL Server.

3.
Click the New Query button in the toolbar to open a new query window.

4.
Enter the following SQL statements:

USE master

CREATE LOGIN [SQL-NS_Dev] WITH PASSWORD='devPassword'
GO
EXEC sp_addsrvrolemember 'SQL-NS_Dev', sysadmin

5.
If you chose a different username, replace SQL-NS_Dev with the username you've chosen. (You'll need to make two replacements.)

6.
Replace devPassword with the password you've chosen. (You need to make only one replacement.)

7.
Execute the query by pressing F5.

Note

You may be tempted to save the query script you executed previously for reuse later. If you do so, make sure that you edit out the password before saving. It's not a good idea to leave passwords lying around in scripts.

The Test Account

Choose a username and password for your test account. On my system, I've chosen the username, SQL-NS_Test, but you may use another username if you want. Be sure to remember the username and password you've chosen; you'll need them later.

Note

These instructions assume you still have Management Studio open (after completing the steps in the previous section to create the development account) and you're logged in to your SQL Server under a system administrator account.

1.
Open a new query window and enter the following SQL statements:

USE master

CREATE LOGIN [SQL-NS_Test] WITH PASSWORD='testPassword'

2.
If you chose a different username, replace SQL-NS_Test with the username you've chosen. (You'll need to make only one replacement.)

3.
Replace testPassword with the password you've chosen. (You need to make only one replacement.)

4.
Execute the query by pressing F5.

Notice that the test account is not granted membership in any fixed server roles. When we create SQL-NS applications in subsequent chapters, we'll grant the test account the specific database permissions it needs.

Environment Variables

The command scripts (batch files) used to compile and run the sample SQL-NS applications in this book rely on a set of environment variables to work correctly. These environment variables specify where the source code is installed, the name of the SQL Server, the name of the account under which the engine should run, and the SQL authentication mode to use.

Each of the batch files that execute SQL-NS commands first calls another batch file, setenv.cmd, that gets these environment variables set up correctly. Setenv.cmd is located in a common directory that every other batch file references by relative path. The following steps describe how to edit setenv.cmd to set up the environment variables appropriately for your environment. In the next section, we'll run a test script that verifies the environment settings you make here and we'll troubleshoot any problems that may occur. Finally, in the "Customizing the Source Files for Your Environment" section (p. 39), you'll run a script that uses the settings you make here to insert environment-specific values into some of the source files you copied off the CD-ROM.

1.
Open a command prompt on your development machine and navigate to the source code base directory (the directory in which you copied the source code from the CD-ROM earlier) by typing the following command:

cd /d C:\SQL-NS

2.
Navigate to the Common subdirectory:

cd Common

3.
Open the setenv.cmd file in Notepad:

notepad setenv.cmd

This batch file sets up environment variables used by other batch files. In the following steps, you'll edit individual lines in this batch file that assign values to the variables.

4.
Find the line that says set SQLNS_BASE_DIRECTORY_PATH= and add the name of the source base directory at the end. For example, if your source base directory is C:\SQL-NS, change the line to read

set SQLNS_BASE_DIRECTORY_PATH=C:\SQL-NS

5.
Find the line that says set SQL_SERVER= and add the name of your SQL Server at the end. For example, my SQL Server runs on a computer called PATHERNET01, so I've edited the line to read

set SQL_SERVER=PATHERNET01

Note

If your SQL Server is local, this is the name of the development machine; otherwise, it is the name of the computer running SQL Server.

If you're using a named (nondefault) instance of SQL Server, the SQL Server name should be of the form <Machine Name>\<Instance Name>for example, MyServer1\MyInstance1.

6.
Find the line that says set NS_SERVICE_USER_NAME=, and add the name of the account you created for the service (as described in the "Creating the Service Account" section, p. 30, earlier in this chapter) at the end. The account I created was called SQL-NS_Service; so, on my system, the line reads

set NS_SERVICE_USER_NAME=SQL-NS_Service

7.
Find the line that says set NS_SERVICE_USER_DOMAIN=. If the account you created for the service user is a domain account, add the name of the domain to the end of this line. If the account you created for the service is a local account, add the name of your development machine to the end of this line. On my system, the account is a domain account, and the name of the domain is PATHERNET, so the line reads

set NS_SERVICE_USER_DOMAIN=PATHERNET

8.
Find the line that says set USE_SQL_AUTH=0. If you are using Windows Authentication, leave this line as it is and skip to step 11. If you are using SQL Server Authentication, change this line so that it reads

set USE_SQL_AUTH=1

9.
Find the line that says set DEV_SQL_USER= and add the name of the SQL development account at the end. My SQL development account is called SQL-NS_Dev,; so, on my system, the line reads

set DEV_SQL_USER=SQL-NS_Dev

10.
Find the line that says set TEST_SQL_USER= and add the name of the SQL test account at the end. My SQL test account is called SQL-NS_Test; so, on my system, the line reads

set TEST_SQL_USER=SQL-NS_Test

11.
Save the file and close it.

Testing Your Development Environment

In the previous section, you edited the batch file that sets up the required environment variables and supplied values appropriate for your system environment. In this section, you'll run a test script that determines whether everything is set up correctly. This section assumes that you have completed the preceding steps.

Note

The test script described here uses the sqlcmd.exe command-line utility to connect to your SQL Server. The script will check whether sqlcmd.exe is in your path and will provide instructions for adding it to your path if necessary. Several of the scripts used in later chapters of this book require sqlcmd.exe to be in your path, so I recommend you set this up now.

You can find sqlcmd.exe in the 90\Tools\Binn directory under the directory in which you have SQL Server 2005 installed (usually %ProgramFiles%\Microsoft SQL Server). In most cases, the SQL Server setup wizard adds this directory to your path automatically when you install the SQL Server client tools, but you may have to add it manually on your system.

1.
Open a command prompt on your development machine and navigate to the source code base directory (the directory in which you copied the source code from the CD-ROM earlier) by typing the following command:

cd /d C:\SQL-NS

2.
Navigate to the Common subdirectory:

cd Common

3.
Run the testenv.cmd batch file.

This batch file attempts to exercise the environment settings you made in the previous section and displays error messages and troubleshooting suggestions if problems occur. If you see any error messages, follow the suggested instructions for resolving the problems. These usually tell you to change one or more environment variable settings in the setenv.cmd file you edited in the previous section. After you've made the appropriate changes, follow the instructions in this section to execute testenv.cmd again.

Repeat this process until testenv.cmd completes without errors. When the testenv.cmd batch file completes successfully, you'll know you're ready to proceed.

Customizing the Source Files for Your Environment

Many SQL-NS source files contain settings and other information that must be customized for each environment. For example, some SQL-NS source files contain directory paths whose values depend on where the source files are installed.

On this book's CD-ROM, source files containing environment-specific information are shipped as templates, with placeholders for the values that are different in each environment. As a final step in getting prepared to work through the examples in this book, you'll need to run a script that customizes the template source files you copied from the CD-ROM. This script uses the environment variable settings you created and tested in the previous two sections to replace the placeholders in the template source files with the correct values for your system. In addition, the script also performs some final configuration steps (such as granting appropriate filesystem permissions to the service account used to run the SQL-NS Windows service).

After you complete the following instructions, the source code on your system will be ready for you to work with (as described in subsequent chapters). It's important that you complete these instructions now; otherwise, the sample applications will not compile.

1.
From the Start menu on your development machine, open a Notification Services Command Prompt by selecting All Programs/Microsoft SQL Server 2005/Configuration Tools/Notification Services Command Prompt.

Caution

The instructions that follow must be completed in a Notification Services Command Prompt. Do not use a regular command prompt window to execute these commands.

2.
Navigate to the source code base directory (the directory in which you copied the source code off the CD-ROM earlier) by typing the following command:

cd /d C:\SQL-NS

3.
Navigate to the Common subdirectory:

cd Common

4.
Run the setup_all.cmd batch file.

Caution

If you're using SQL Server Authentication, you will receive several prompts for SQL passwords as setup_all.cmd runs. In each case, pay attention to the account for which the password is requested. Most of the prompts request the password for the development account, but a few require the password for the test account.

When you run setup_all.cmd, you may see some error messages because the script performs cleanup operations that try to remove things that may not be present on your system the first time you run it. You can safely ignore these error messages, as long as the script prints a final message, Setup SUCCEEDED, at the very end.

Summary

In this chapter, you installed the required software and set up your development environment to work with the source code in the rest of this book. When you start building your own notification applications, you'll be able to use this same development environment to perform SQL-NS development and administration tasks.

In setting up the environment, you had to pay particular attention to security and user permissions. Although these steps can sometimes seem tedious, I urge you to continue being diligent about doing them when building and deploying your own applications.


Chapter 3. The Simplest Notification Application: Stock Quotes

IN THIS CHAPTER

  • The SQL-NS Application Model

  • Building the Stock Application's ADF

  • Specifying Other Parts of the Stock Application

  • Running the Stock Application

  • Inside the Running Application

  • What Has the SQL-NS Platform Provided?

  • Cleaning Up the Instance and Application

In this chapter, you'll see your first SQL-NS application: a stock notification service similar to those offered by many real-world stockbrokers. The application allows subscribers to enter subscriptions for stocks in which they are interested and notifies them when those stocks cross the price targets they specify.

Think of this chapter as a tour. My intent is simply to show you around the various facilities that the SQL-NS platform offers so that you get a feel for the application model and the process of coding to it.

We will look at code in this chapter, but simply for the purpose of understanding the concepts behind the application. A line-by-line explanation of the code at this stage would drown out the simpler picture that I'm trying to show.

Instead, I will gloss over (or in some cases, completely ignore) some parts of the code you will see; I'll just highlight those pieces of the code that illustrate particularly important parts of the application model. Subsequent chapters will cover the rest in detail.


The SQL-NS Application Model

SQL-NS can be used to build a variety of notification applications with different uses and for different application domains. But, when viewed from the highest level, all these notification applications conform to the same basic model, shown in Figure 3.1.

Figure 3.1. High-level view of a notification application.

Data enters the application from the outside world. This data can be pulled in by the application or pushed in by an external source. In SQL-NS terms, each piece of data is referred to as an event because it represents some happening in the outside world that may potentially be of interest to some subscribers. An event may be a new price for a stock, a notice of a new concert listing, or a gate change for a flight.

The notification application maintains users' subscriptions. Subscriptions are users' declarations of what kinds of events interest them. When events arrive, the application matches them against the subscriptions and produces a set of notifications. These notifications are delivered to the end users.

Events as Data

Events are descriptions of things that happen in the real-world that can be represented as data. For example, a change in the price of a stock (an event potentially of interest to a stockbroker client) can be described as a piece of structured data containing a stock symbol field and a stock price field. A traffic incident event (for an application that notifies commuters about road conditions) might contain a field that describes the location of the incident and another describing the incident type (accident, road closure, weather warning, and so on).

Whatever the type of event, its description can be modeled as data. The structure of the data can be described with a schema that indicates the names of the fields and their data types. Given this schema, it's easy to construct a database table to store the event data. For example, stock events can be stored as rows in a table, as shown in Figure 3.2.

Figure 3.2. Modeling stock events as rows in a table.

Stock Symbol | Stock Price
XYZ | 55.55
PQS | 95.30
JKL | 15.00

Subscriptions as Data

Thinking of events as data is usually quite natural. Although it may be somewhat less intuitive at first, subscriptions can be modeled as data, too. Think of the subscriptions in a stock notification application. Let's say that all subscriptions will have the form

"Notify me when the price of stock S reaches price target T."

where S represents some stock symbol and T represents a price target. A subscription example might be

"Notify me when the price of stock XYZ reaches price target $50.00."

If all the subscriptions are constrained to this form, an individual subscription can be represented as a pair of values for S and T. Such subscriptions could be stored in a table, as shown in Figure 3.3.

Figure 3.3. Modeling stock subscriptions as rows in a table.

Subscriber | Stock Symbol (S) | Price Target(T)
Bob | XYZ | 50.00
Alex | PQS | 90.00
Mary | JKL | 12.00
Jane | PQS | 100.00

Each row identifies the subscriber, the stock symbol, and the price target of interest. For illustrative purposes, the subscriber is represented by a name, but in an actual application, a more appropriate identifier may be used.

Matching Events with Subscriptions

As mentioned in Chapter 1, "An Overview of Notification Applications," the matching of events against subscriptions is the key function of any notification application. If the matching can be implemented efficiently, the application will scale to large volumes.

With events and subscriptions both represented as data, matching can be accomplished by means of a SQL join. Given the table structures in Figures 3.2 and 3.3 for events and subscriptionsand let's say that we called the events table Events and the subscriptions table Subscriptionsthe following SQL statement would determine the matches:

SELECT  S.Subscriber, E.StockSymbol, E.StockPrice
FROM    Events E JOIN Subscriptions S
ON      E.StockSymbol = S.StockSymbol
WHERE   E.StockPrice >= S.PriceTarget

This statement joins the stock events table with the subscriptions table on stock symbol and then selects rows where the stock price in the event is greater than or equal to the stock price target specified in the subscription. The rows returned by this query represent the set of matches between the events and subscriptions that should result in notifications being sent. For the particular data shown in the preceding examples, Figure 3.4 shows the results of the query.

Figure 3.4. Results of matching events with subscriptions.

Subscriber | Stock Symbol | Stock Price
Bob | XYZ | 55.55
Alex | PQS | 95.30
Mary | JKL | 15.00

Note that only three of the four subscriptions matched: Jane's subscription specified a price target for PQS of 100.00, and because the event for PQS indicated that the price was only 95.30, the matching query did not return a row for Jane.

Each of the rows in the results table is the raw data for a notification to be sent. Thus, notifications also can be modeled as rows of data in a table. The notification data can later be packaged into a readable message and delivered to the appropriate subscriber.

Scalability of the SQL-NS Application Model

The modeling of both events and subscriptions as data is a key innovation of SQL-NS and the basis for its inherent scalability. Because both events and subscriptions are rows in tables, SQL joins can be used to match them. In general, SQL joins are extremely efficient at matching large sets of data; more than 20 years of query processing and indexing developments make this possible. As long as a reasonable join query can be written for a particular event and subscription schema (in most cases, one can), the cost of matching (in terms of computing resources) will be low. Furthermore, this cost grows sublinearly with the amount of data. That is, if you double the number of events or subscriptions, the cost of matching increases by less than a factor of two.

This model is different from that used by most other pub-sub systems. Most other systems model individual subscriptions as queries, rather than data. The simplest of these systems evaluates the subscription queries one-by-one for a given set of events. This strategy is expensive, and as the number of subscriptions and events grows, the cost of evaluation growsat best, linearly, at worst, exponentially. The more sophisticated of these systems attempt to obtain performance gains by indexing the subscription queries and looking for logical shortcuts in the query evaluation. But the effectiveness of these strategies is limited by the structural differences and complexity of the subscriptions and isn't always assured. In many cases, systems that model subscriptions as queries can do little better than one-at-a-time evaluation.

Note

To be absolutely clear, the SQL-NS application model does use queries to evaluate subscriptions. But it does not model each individual subscription as a query. Instead, in the SQL-NS application model, there is one query for each subscription type. This query evaluates all subscriptions of that type at once. This is the key differentiator between SQL-NS and other pub-sub systems.

When you build a SQL-NS application, you have a choice of two models for subscription evaluation. In the first model, developer-defined logic is used to determine whether a match between an event and a subscription has occurred. In this model, you, the application developer, write the SQL join query that finds the matches. This query might look something like the SQL statement shown earlier in the "Matching Events with Subscriptions" section (p. 43).

By writing the query, you are defining the conditions that must be true for a match to occur. The subscribers can specify only values for the data referenced in the query. For example, the logic coded into the stock application's query says that a stock event matches a stock subscription when the stock symbols are the same and the stock price in the event is greater than, or equal to, the stock price target in the subscription. Subscribers cannot change this logic; they can only provide values for the stock symbol and stock price target in individual subscriptions.

In the second subscription evaluation model, user-defined logic determines whether an event matches a subscription. The users of the application (the subscribers) choose the conditions that must be true for a match to occur. These conditions can be arbitrary combinations of Boolean predicates based on the event data. In this model, each subscription can specify a different matching condition. For example, given the stock event data, one subscription's condition might state that a match occurs when the stock price in the event is less than a particular price target. Another subscription's condition may stipulate that a match occurs when the stock price in the event is within a particular range of values.

Regardless of whether you choose developer-defined or user-defined logic for the subscriptions in your application, SQL-NS uses queries to evaluate groups of subscriptions at the same time. In the case of developer-defined logic, the query provided by the developer is associated with a named subscription type, and all subscriptions of that type are evaluated by the single query. When user-defined logic is used, SQL-NS also uses a single query to evaluate subscriptions of a common type. The difference is that subscriptions are grouped into types based on the logical structure of their conditions (rather than assigned to predefined, named subscription types), and the queries used to evaluate them are constructed dynamically by SQL-NS. For a description of how SQL-NS uses queries to evaluate subscriptions with user-defined logic, refer to the "Evaluating User-Defined Logic with Queries" sidebar (p. 45).

Evaluating User-Defined Logic with Queries

To evaluate user-defined logic, SQL-NS begins by translating each subscription into a SQL query that selects from the events table the rows that satisfy the subscription condition. The condition logic specified in the subscription is implemented in the WHERE clauses of this query. For example, suppose a subscriber, Bob, wants to be notified when the XYZ stock price falls between 45 and 75. He might enter a subscription with the following condition:

[View full width]

(E.StockSymbol = 'XYZ') AND ((E.StockPrice > 45) AND (E
  .StockPrice <= 75))

This could be translated into the following SQL query:

SELECT  'Bob' AS Subscriber, E.StockSymbol, E.StockPrice
FROM    Events E
WHERE   E.StockSymbol = 'XYZ'
        AND (E.StockPrice > 45 AND E.StockPrice <= 75)

When two or more subscription conditions translate into SQL queries with the same structurethat is, queries with the same combinations of logical constructs that differ only in terms of constant data values, SQL-NS builds a single query to evaluate them. That query does not contain hard-coded data values (as the one in the preceding example does), but it references a parameters table that stores the specific constant values for each subscription. For example, suppose that in addition to Bob's subscription we just examined, another subscriber, Mary, wants to be notified when the PQS stock price falls between 80 and 120. Mary's subscription condition

[View full width]

(E.StockSymbol = 'PQS') AND ((E.StockPrice > 80) AND (E
  .StockPrice <= 120))

could be translated into the following query:

SELECT  'Mary' AS Subscriber, E.StockSymbol, E.StockPrice
FROM    Events E
WHERE   E.StockSymbol = 'PQS'
        AND (E.StockPrice > 80 AND E.StockPrice <= 120)

But because this query is structurally the same as the query that evaluates Bob's subscription, both subscriptions could be evaluated together by the following single query:

[View full width]

SELECT  P.Subscriber, E.StockSymbol, E.StockPrice
FROM    Events E, SubscriptionParameters P
WHERE   E.StockSymbol = P.Parameter1
        AND (E.StockPrice > P.Parameter2 AND E.StockPrice <= P
  .Parameter3)

This query relies on the SubscriptionParameters table to provide the constant data values for each subscription: for Bob's subscription, the values are the stock symbol, XYZ, and the price targets, 45 and 75; for Mary's subscription, the stock symbol is PQS, and the price targets are 80 and 120. To evaluate additional subscriptions that have the same logical form, new rows are needed in the parameters table, but the same query can evaluate them all together.

Because the same fundamental condition can be expressed in many ways using Boolean constructs, SQL-NS attempts to normalize the condition statements before translating them into SQL queries. This maximizes the opportunity for the same query to be used for multiple subscriptions.

Note that the queries and table structures shown here are simplified for ease of explanation. In actual applications, the queries and supporting tables generated by SQL-NS are somewhat more complex because they are designed to accommodate an unlimited number of parameters per query and support many different data types. However, they are based on the basic principles described here.

Your choice of subscription-evaluation model (developer-defined or user-defined logic) affects the scalability of your applications. Generally, applications that use developer-defined logic are more scalable because subscription grouping is enforced by the developer. With user-defined logic, the extent to which grouping is possible depends on the similarity of the subscriptions entered. In the worst case, if every subscription condition has a different structure, no grouping is possible and the application would have to evaluate one subscription at a time. In practice, it's unlikely that this worst-case scenario would actually play out; in most cases, some degree of grouping is achievable.

Just because you have the option of supporting user-defined logic in your SQL-NS applications does not mean you always should. It's worth questioning whether the flexibility offered by user-defined logic, which requires a sacrifice in scalability, is really needed in your applications. The obvious drawbacks to using developer-defined logic are that all subscriptions must have the same structure, and the only kinds of subscriptions users can enter are those the developer has implemented. But do your users really need to be able to enter arbitrarily complex subscriptions? Does each user need to be able to enter a different kind of subscription? Or can you devise a set of subscription types that cover the overwhelming majority of subscriptions users will want to enter?

Remember that SQL-NS enables you to implement several types of subscriptions in a single application, each with a separate data schema and matching query. Experience has shown that in the vast majority of applications, developers can predict what subscriptions users will want to enter and implement those as predefined subscription types. Often, the lack of flexibility in the subscription-matching logic is not even noticed by the users of these applications.

All the sample applications in Parts I, II, and III of this book (including the sample application in this chapter) use only developer-defined matching logic. The use of user-defined matching logic is covered in Part IV, in Chapter 18, "User-Defined Matching Logic in SQL-NS Applications." At this point, you should focus on learning the fundamental SQL-NS concepts, the majority of which are the same, regardless of which subscription evaluation model you choose.

Programming to the SQL-NS Application Model

In essence, the SQL-NS application model views events and subscriptions as data and uses SQL joins to match them. As a developer building on the SQL-NS platform, you define the following aspects of your application:

  • Schemas for the event, subscription, and notification data

  • Logic that the application executes to perform matching and maintain state

  • Configuration of the SQL-NS execution engine components that run the application

You provide all this information in an XML document called an Application Definition File (ADF). Think of the ADF as the source code for the application: to create a new application, you author a new ADF.

Note

SQL-NS provides an XSD schema that defines the XML elements an ADF may contain. As you work through this and subsequent chapters of this book, you'll learn about these XML elements in detail. This chapter highlights specific pieces of the stock notification application's ADF.

When you write an ADF, you usually begin with the elements that define schemas. Each schema is a description of the size and shape of one kind of data. You provide the names of the fields and their data types, much as you would if you were defining a SQL table. The schema for the events specifies the structure of the data your application receives from its event sources. The schema for the subscription data describes the information each subscriber will provide when creating a subscription. The notification schema describes the data content of the notifications your application will deliver.

In addition to the data schemas, the ADF also specifies how events and subscriptions are matched to form notifications. If you choose developer-defined logic as your subscription evaluation model, the matching logic you provide in the ADF consists of actual SQL statements that find the matches. These statements operate on the event and subscription data (as defined by their respective schemas) and produce a row of data for each notification to be sent (the columns in the resultsets correspond to the fields in the notification schemas). If instead you choose user-defined logic as your evaluation model, in the logic sections of the ADF, you provide SQL statements that produce notification data (conforming to the notification schemas) from event and subscription data that has already been determined to match (according to the user-defined conditions).

In the component configuration section of the ADF, you specify how the SQL-NS engine components should run your application. These components perform such functions as gathering event data from event sources, coordinating the execution of the matching logic, and delivering notifications to their destinations. You can specify which components should be used in your application, how they are distributed across various servers, what resources they should use, and when they should run.

When your ADF is complete, you compile it using the SQL-NS compiler. The compiler translates the XML application definition into a set of database objects that will be used to run the application. These include tables for configuration settings, events, subscriptions, and notifications, as well as stored procedures that execute the SQL statements you provided in the ADF. The compiler installs these database objects into a database on your SQL server and populates some of the tables with the configuration information from the ADF. When you run your application, the SQL-NS engine connects to this database, reads the configuration information, and starts its various components; those components then interact with the database (reading and writing data) as they perform their execution functions. This process is illustrated in Figure 3.5.

Figure 3.5. The ADF is compiled into database structures, and the SQL-NS engine runs the application.

Note

Some important parts of a complete, running applicationsuch as the event sources, the subscription management interface, and the delivery systemsare not shown in Figure 3.5. Usually, these other parts of the application also interact with the application database, alongside the SQL-NS engine.

In the remainder of this chapter, we'll create and run the stock notification application. In doing so, we'll look at the data schemas, matching logic, and component configuration, as specified in the application's ADF. Although there are other aspects to the application, we'll focus on these for now because they form the application's core and are the most illustrative of the SQL-NS application model.

Note

To work through the steps in this chapter, you need to set up your development environment as described in Chapter 2, "Getting Set Up." If you have not already done so, go through the steps in that chapter before proceeding.


Building the Stock Application's ADF

The ADF for the stock application defines its schemas and logic, as well as its component configuration. The schemas describe the size and shape of events, subscriptions, and notifications; the logic defines how events and subscriptions are matched to produce notifications. The component configuration tells the SQL-NS engine how to actually run the application.

Table 3.1 shows the high-level structure of the ADF (with the application-specific details removed). Think of this as a basic outline for every ADF you will ever write. A root <Application> element contains various other XML elements that define the parts of the application. This section describes the content that goes in each of these XML elements to create a real application.

Table 3.1. Basic Structure of the ADF

XML Element

Purpose

<?xml version= "1.0" encoding="utf-8" ?>

Standard XML header

[View full width]

<Application
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns
  :xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="...">

Root element that contains all of the application definition

<Database>

</Database>

Specifies the database in which the application's tables, views, stored procedures, and other objects should be created

<EventClasses>

</EventClasses>

Defines schemas for the events the application will receive

<SubscriptionClasses>

</SubscriptionClasses>

Defines schemas and matching logic for the subscriptions the application will support

<NotificationClasses>

</NotificationClasses>

Defines schemas for the notifications the application will send

<Providers>

</Providers>

Configures the application's event providers

<Generator>

</Generator>

Configures the generator component that matches events with subscriptions

<Distributors>

</Distributors>

Configures the distributor components that deliver notifications

<ApplicationExecutionSettings>

</ApplicationExecutionSettings>

Contains operational settings that control various aspects of the application's behavior

</Application>

Closing tag for the root element

Note

I've left some rarely used ADF elements out of Table 3.1 for clarity. These include the <History> and <Version> elements, which are intended primarily for developers to track changes to source files and do not affect the operation of the application in any way. These elements are described in the SQL-NS Books Online. Also, some of the elements in Table 3.1 are optional; they will not be present in every ADF. Later chapters cover each element in detail and explain the defaults used when optional elements are omitted.

Note

In SQL Server 2005, the SQL-NS Books Online are included as part of the SQL Server Books Online. To open the SQL Server Books Online, go to the Start menu, All Programs, Microsoft SQL Server 2005, Documentation and Tutorials, SQL Server Books Online. The SQL-NS-specific documentation can be found in the "SQL Server Notification Services" topic in the main table of contents.

The <EventClasses>, <SubscriptionClasses>, and <NotificationClasses> subelements under <Application> contain all the schemas and logic. The other elements provide configuration information used by the SQL-NS compiler and the SQL-NS engine.

The Completed ADF

Before delving into the details of each specific section, take a look at the completed ADF, shown in Listing 3.1. I'm including this here, even before I explain what any of it means, because I find that with almost any program, it helps to get a feel for the code by looking at it from end to end. Just by glancing at it, the ADF will probably make some sense to you, even without further explanation. The following sections describe the important parts in detail.

Listing 3.1. The Completed Stock ADF

[View full width]
<?xml version="1.0" encoding="utf-8" ?>
<Application xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:xsi=http://www.w3.org/2001
  /XMLSchema-instance xmlns= "http://www.microsoft.com/MicrosoftNotificationServices
  /ApplicationDefinitionFileSchema">

  <Database>
    <DatabaseName>StockBroker</DatabaseName>
    <SchemaName>StockWatcher</SchemaName>
  </Database>

  <EventClasses>
    <EventClass>
      <EventClassName>StockPriceChange</EventClassName>
      <Schema>
        <Field>
          <FieldName>StockSymbol</FieldName>
          <FieldType>NVARCHAR(10)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
        <Field>
          <FieldName>StockPrice</FieldName>
          <FieldType>SMALLMONEY</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
    </EventClass>
  </EventClasses>

  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>StockPriceHitsTarget</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>StockSymbol</FieldName>
          <FieldType>NVARCHAR(10)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
        <Field>
          <FieldName>StockPriceTarget</FieldName>
          <FieldType>SMALLMONEY</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
      <EventRules>
        <EventRule>
          <RuleName>MatchStockPricesWithTargets</RuleName>
          <Action>
            INSERT INTO [StockWatcher].[StockAlert]
            SELECT subscriptions.SubscriberId,
                   N'DefaultDevice',
                   N'en-US',
                   events.StockSymbol,
                   events.StockPrice,
                   subscriptions.StockPriceTarget
            FROM   [StockWatcher].[StockPriceChange] events
            JOIN   [StockWatcher].[StockPriceHitsTarget] subscriptions
                ON events.StockSymbol = subscriptions.StockSymbol
            WHERE  events.StockPrice &gt;= subscriptions.StockPriceTarget
          </Action>
          <EventClassName>StockPriceChange</EventClassName>
        </EventRule>
      </EventRules>
    </SubscriptionClass>
  </SubscriptionClasses>
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>StockAlert</NotificationClassName>
      <Schema>
        <Fields>
          <Field>
            <FieldName>StockSymbol</FieldName>
            <FieldType>NVARCHAR(10)</FieldType>
          </Field>
          <Field>
            <FieldName>StockPrice</FieldName>
            <FieldType>SMALLMONEY</FieldType>
          </Field>
          <Field>
            <FieldName>StockPriceTarget</FieldName>
            <FieldType>SMALLMONEY</FieldType>
          </Field>
        </Fields>
      </Schema>
      <ContentFormatter>
        <ClassName>XsltFormatter</ClassName>
        <Arguments>
          <Argument>
            <Name>XsltBaseDirectoryPath</Name>
            <Value>%_ApplicationBaseDirectoryPath_%\XslTransforms</Value>
          </Argument>
          <Argument>
            <Name>XsltFileName</Name>
            <Value>StockAlert.xslt</Value>
          </Argument>
        </Arguments>
      </ContentFormatter>
      <Protocols>
        <Protocol>
          <ProtocolName>File</ProtocolName>
        </Protocol>
      </Protocols>
    </NotificationClass>
  </NotificationClasses>

  <Providers>
    <NonHostedProvider>
      <ProviderName>TestEventProvider</ProviderName>
    </NonHostedProvider>
  </Providers>

  <Generator>
    <SystemName>%_NSServer_%</SystemName>
  </Generator>

  <Distributors>
    <Distributor>
      <SystemName>%_NSServer_%</SystemName>
      <QuantumDuration>PT15S</QuantumDuration>
    </Distributor>
  </Distributors>

  <ApplicationExecutionSettings>
    <QuantumDuration>PT15S</QuantumDuration>
  </ApplicationExecutionSettings>

</Application>

Before you continue reading, you should open the stock application's source code on your system so that you can browse the ADF as you go. Use the following instructions to open the project in Management Studio and bring up the ADF in the built-in XML editor:

1. | Start Management Studio (from the Start menu, choose All Programs, SQL Server 2005, Microsoft SQL Server Management Studio) and connect to your SQL Server.
2. | From the File menu, choose Open, Project/Solution.
3. | In the File Open dialog box, browse to the C:\SQL-NS\Samples\StockBroker directory and select the StockBroker.ssmssln solution file.
4. | If the Solution Explorer window is not visible, open it by selecting Solution Explorer from the View menu or by pressing Ctrl+Alt+L.
5. | In the Solution Explorer, you should see two projects: StockBroker and StockWatcher. Expand the StockWatcher project node and the Miscellaneous folder beneath it.
6. | Open the ApplicationDefinition.xml file.

The Database Element in the ADF

As described earlier in the "Programming to the SQL-NS Application Model" section (p. 47), the SQL-NS compiler creates database objects (including tables, views, and stored procedures) based on the information in the ADF. You can choose where these database objects are installed: in the <Database> element of the ADF, you can provide a target database name and a schema name. When you compile the ADF, the resulting database objects are installed in the database and schema you specify (see the sidebar titled "User-Schema Separation in SQL Server 2005," p. 55, for an explanation of the term schema in this context).

In the case of the stock application's ADF, shown in Listing 3.1, the <Database> element specifies a database name as StockBroker and the schema name as StockWatcher. I chose these names arbitrarily. StockBroker seemed like a fitting name for a database related to stockbroker operations and StockWatcher aptly describes the purpose of this application. All database objects for this application created by the SQL-NS compiler will be placed in the StockWatcher schema in the StockBroker database. In the later sections of this chapter, you'll see these database objects referred to by their schema-qualified names.

Note

If the database and schema specified in the <Database> element don't exist at compilation time (as will likely be the case on your system), the SQL-NS compiler will create them.

User-Schema Separation in SQL Server 2005

"Schema" is one of the most overloaded words in database terminology. It generally refers to a description of the structure and organization of some kind of data. However, in SQL Server 2000 and SQL Server 2005, schema also refers to a namespace for a group of related database objects. This sidebar explains this usage of the term. Wherever the word schema is used in the remainder of this book, I explicitly call out the intended meaning, if it isn't obvious from the context.

In SQL Server 2000, schemas were used to group objects in a database created by a single user. Each user had an associated schema (the schema name was the same as the username). A user's schema isolated the database objects she created from those created by other users. For example, if user Jane created a table called Products, the table would, by default, go into the Jane schema. Its schema-qualified name would be "Jane.Products". This would allow another userfor example, Joeto also create a Products table, without resulting in a name collision (Joe's table would be called "Joe.Products").

In SQL Server 2005, schemas are separated from users. You can create schemas independently of users, simply to group related database objects. For example, in a given database, you can create a schema called Inventory, and then create a table within this schema called Products. The schema-qualified name for this table would then be "Inventory.Products". You can also create other schemas and place Products tables in those schemas without name collisions. For example, you can create a Manufacturing schema, with a "Manufacturing.Products" table.

Schema-qualified names can be used in any place nonqualified names are normally used. For example, to write a query that refers to tables in specific schemas, you would include the schema-qualified table names in the FROM clause.

Schemas and Logic

This section describes the ADF syntax used to define the event, subscription, and notification data schemas (here I'm referring to the usual meaning of schemasthe definition of the data's structure). This section also shows how the matching logic, expressed as a SQL join, is specified in the ADF.

Event Schemas

For each type of event that the application receives (there can be more than one), you must declare an event class in the <EventClasses> element of the ADF.

The stock application uses only a single type of event: a change in the trading price of a stock. Within the <EventClasses> element of the ADF, you declare an <EventClass> for this type of event, as shown in Listing 3.2.

Listing 3.2. Declaration of the StockPriceChange Event Class

<EventClass>
  <EventClassName>StockPriceChange</EventClassName>
  <Schema>
    <Field>
      <FieldName>StockSymbol</FieldName>
      <FieldType>NVARCHAR(10)</FieldType>
      <FieldTypeMods>NOT NULL</FieldTypeMods>
    </Field>
    <Field>
      <FieldName>StockPrice</FieldName>
      <FieldType>SMALLMONEY</FieldType>
      <FieldTypeMods>NOT NULL</FieldTypeMods>
    </Field>
  </Schema>
</EventClass>

The declaration provides a name, StockPriceChange, for the event class and a schema for the event data. The schema declaration syntax is basically an XML version of a SQL CREATE TABLE statement. It declares a set of fields specifying a data type and, optionally, a type modifier (such as not null) for each.

When building the application's database, the SQL-NS compiler processes this event class declaration and constructs a table that will ultimately store the events. It creates a column for each of the fields declared in the event class and adds some other columns used for internal tracking.

The SQL-NS compiler also builds a view over the events table. The columns in the view match the fields declared in the event class exactlythe extra columns created in the events table for SQL-NS internal tracking are not present in the view. This view becomes the primary means by which events are manipulated in the application. To submit event data for processing, you insert rows into the view. Triggers on the view (also created by the SQL-NS compiler) handle inserting the appropriate values into the internal columns in the events table. When matching against subscriptions, event data is read from the view.

Subscription Schemas

Just as you have to declare an event class for each type of event the application receives, you also have to declare a subscription class for each type of subscription that the application supports. This simple stock application has only one type of subscription: a request to be notified when a stock hits a particular price target. In the completed ADF, this subscription class declaration goes within the <SubscriptionClasses> element. Listing 3.3 shows the subscription class declaration.

Listing 3.3. Declaration of the StockPriceHitsTarget Subscription Class

<SubscriptionClass>
  <SubscriptionClassName>StockPriceHitsTarget</SubscriptionClassName>
  <Schema>
    <Field>
      <FieldName>StockSymbol</FieldName>
      <FieldType>NVARCHAR(10)</FieldType>
      <FieldTypeMods>NOT NULL</FieldTypeMods>
    </Field>
    <Field>
      <FieldName>StockPriceTarget</FieldName>
      <FieldType>SMALLMONEY</FieldType>
      <FieldTypeMods>NOT NULL</FieldTypeMods>
    </Field>
  </Schema>
  <EventRules>
    ...
  </EventRules>
</SubscriptionClass>

The subscription class declaration has three subelements: a name, a schema, and a set of event rules (content is not shown in the fragment in Listing 3.3).

The schema subelement defines the size and shape of the subscription data. In this application, we're using developer-defined matching logic and constraining subscriptions to the form

"Notify me when the price of stock S goes above price target T."

The subscription data for each subscription then just consists of values for S and T. In the subscription class schema, we've given these fields the more descriptive names StockSymbol and StockPriceTarget and provided their data types and type modifiers.

Just as it does with the event class schema declaration, the SQL-NS compiler constructs a table for subscription data from the subscription class schema declaration. Like the events table, the subscriptions table contains a column for each field declared and some extra columns used internally by SQL-NS. Also, the SQL-NS compiler constructs a view over the subscriptions table, with columns matching the declared subscription class fields. When you need to add subscriptions of this subscription class, you insert rows into the subscriptions view. When you define the logic that matches subscriptions against events, you access subscription data by selecting from the subscriptions view.

The matching logic for the subscription class is declared in the <EventRules> subelement. This is discussed in the "Matching Logic" section (p. 59).

Notification Schemas

The <EventClasses> and <SubscriptionClasses> elements define the schemas for the application's events and subscriptions. When the matching logic is applied, the result is a set of notification data that represents the notifications to be sent to subscribers. This notification data has a schema as well and must be declared in the <NotificationClasses> element of the ADF. This section can contain definitions for several types of notifications, but because this stock application example just sends one type of notification, there is just a single <NotificationClass> declaration, shown in Listing 3.4.

Listing 3.4. Declaration of the StockAlert Notification Class

<NotificationClass>
  <NotificationClassName>StockAlert</NotificationClassName>
  <Schema>
    <Fields>
      <Field>
        <FieldName>StockSymbol</FieldName>
        <FieldType>NVARCHAR(10)</FieldType>
      </Field>
      <Field>
        <FieldName>StockPrice</FieldName>
        <FieldType>SMALLMONEY</FieldType>
      </Field>
      <Field>
        <FieldName>StockPriceTarget</FieldName>
        <FieldType>SMALLMONEY</FieldType>
      </Field>
    </Fields>
  </Schema>
  <ContentFormatter>
    ...
  </ContentFormatter>
  <Protocols>
    ...
  </Protocols>
</NotificationClass>

Much like in the event and subscription class declarations, the <Schema> element provides the names and data types of the notification fields. The <ContentFormatter> section describes how the notification data is formatted for receipt by the subscriber, and the <Protocols> section declares which delivery protocols can be used to actually send the notifications. The <ContentFormatter> and <Protocols> elements are discussed more thoroughly in Chapters 5, 9, and 10.

The StockAlert notification schema has three fields: the stock symbol, stock price, and stock price target. Each row of notification data produced by the matching join contains a value for each of these fields. The stock price field contains the current price of the stock (as indicated by the stock event), and the stock price target field contains the price target specified in the subscription. From this notification data, the application can synthesize formatted stock alert messages for delivery to the subscribers, such as

"XYZ is now trading at: $55.55. This is greater than or equal to the target price of $50.00."

As you might expect, the SQL-NS compiler constructs a table for the notification data (based on the declared schema) and a view over this table. The output of the matching logic query is inserted into the notifications view to queue notifications for delivery.

Matching Logic

The stock application's matching logic is specified in the <EventRules> section of the subscription class. This section contains one or more SQL statements that SQL-NS executes when events arrive. Each of these SQL statements is called a rule and is declared in an <EventRule> element. Each rule specifies a name, an action (the SQL query to execute), and the name of the event class that triggers it.

Listing 3.5 shows the <EventRules> element of the StockPriceHitsTarget subscription class. It declares a single rule, MatchStockPricesWithTargets, that simply matches incoming events with StockPriceHitsTarget subscriptions. The <EventClassName> element of the rule declaration specifies the name of the triggering event classin this case, StockPriceChange. This instructs the SQL-NS execution engine to fire this event rule whenever events of the StockPriceChange event class arrive.

Listing 3.5. Event Rules Declaration Within the StockPriceHitsTarget Subscription Class

<EventRules>
  <EventRule>
    <RuleName>MatchStockPricesWithTargets</RuleName>
    <Action>
      ...
    </Action>
    <EventClassName>StockPriceChange</EventClassName>
  </EventRule>
</EventRules>

The matching logic query is specified in the rule's <Action> element. Listing 3.6 shows the contents of the <Action> element from the stock application's ADF.

Listing 3.6. The SQL Matching Logic from the Event Rule's <Action> Element

INSERT INTO [StockWatcher].[StockAlert]
SELECT subscriptions.SubscriberId,
       N'DefaultDevice',
       N'en-US',
       events.StockSymbol,
       events.StockPrice,
       subscriptions.StockPriceTarget
FROM   [StockWatcher].[StockPriceChange] events
JOIN   [StockWatcher].[StockPriceHitsTarget] subscriptions
    ON events.StockSymbol = subscriptions.StockSymbol
WHERE  events.StockPrice &gt;= subscriptions.StockPriceTarget

This logic is just a SQL join that produces a set of rows that becomes notification data.

The first thing to look at in this query is the FROM clause. It selects data from the events view joined with the subscriptions view. Notice that the names of these views are the names of the event class and the subscription class, respectively (with the StockWatcher schema qualifiers). The aliases, events and subscriptions, have been assigned to these views for clarity (these aliases are not mandated by SQL-NS).

The event and subscription views, StockPriceChange and StockPriceHitsTarget, can be thought of as the SQL-NS platform's public interface to the event and subscription data collected. The platform guarantees that these views will contain only the event and subscription data against which the rule should operate, at the time the rule is invoked. In other words, the events view will contain only the events just submitted that have triggered the rule firing, and the subscriptions view will contain only the active subscriptions. (In this simple application, all subscriptions are active, but you'll see in Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State," and Chapter 7, "The SQL-NS Subscription Management API," that subscriptions can be disabled or scheduled to fire only at certain times.)

Given the guarantee that the views always contain only the relevant data when the rule is invoked, the matching query in the ADF doesn't require any custom logic to determine which portions of the data are in scope. This is a great benefit of using SQL-NS: the application logic can be written at a high level of abstraction, leaving the platform to take care of details such as managing the data against which the logic executes.

Note

The scoping of data in the event and subscription views happens because the view definitions refer to internal control tables (set up by SQL-NS at runtime) to select only the relevant rows from the underlying event and subscription tables.

Going back to the query in Listing 3.6, the WHERE clause defines a filter that selects only the rows in which the stock price in the event is greater than or equal to the stock price target in the subscription. Note that the XML escape sequence &gt; is used in place of the > character because this statement appears within an XML document. Use of the > character directly would prevent the document from being well-formed XML.

The join defined in the FROM clause, along with the filter defined in the WHERE clause, implement the matching criteria; the logic specified in the query defines what it means for an event to match a subscription. (The stock symbols must be the same, and the stock price greater than or equal to the price target.) This is an example of developer-defined logic; users cannot change this definition of what a match means.

The results obtained when the matching query is executed determine what notifications will ultimately be delivered. Each row in the resultset represents one notification. Notice that the results of the SELECT query are inserted into the notification class view (which, as you might expect, has the same name as the notification class, StockAlert, defined in Listing 3.4). Inserting rows into a notification class view causes notifications of that notification class to be generated. As is the case for event and subscription classes, the view is the SQL-NS platform's interface to the notification class. Rows inserted into the notification class view will eventually be picked up by the SQL-NS distributors for final formatting and delivery.

The notification class view has a column for each field declared in the notification class. Given the definition of the notification class schema in Listing 3.4, the StockAlert view has columns for StockSymbol, StockPrice, and StockPriceTarget. SQL-NS also adds three other columns to every notification class view:

  • SubscriberId Specifies the ID of the subscriber to receive the notification.

  • DeviceName Specifies the name of the subscriber's device to which the notification should be sent. (Subscriber devices are discussed in more detail in Chapter 7 and Chapter 10, "Delivery Protocols.")

  • SubscriberLocale Specifies the locale for which the notification data should be formatted.

The SELECT clause in Listing 3.6 provides a value in the resultset for each of the columns in the notification class view. The subscriber ID is obtained from the subscription, the device name and locale are constants, two of the notification fields come from data in the event, and the third comes from data in the subscription.

Component Configuration and the Phases of Processing

Previous sections examined the event, the subscription, and the notification schemas and the associated rule logic. This section examines the component configuration elements of the ADF. Specifically, these include the <Providers>, <Generator>, <Distributors>, and <ApplicationExecutionSettings> elements.

Each of the components configured in the ADF plays a specific role in the functioning of the application. Before looking at these components and how they're configured, it's important to understand the processing that happens inside a notification application.

SQL-NS separates the functions of a notification application into separate processing phases:

  • Event collection Gathering events and submitting them to the application

  • Subscription management Creation, deletion, and alteration of subscriptions

  • Generation Matching events with subscriptions

  • Distribution Routing notifications to delivery systems

Figure 3.6 shows these phases.

Figure 3.6. Phases of processing in notification applications.

Each phase is considered independent and is handled by a separate component in the execution engine. The SQL-NS engine may run all these phases concurrently. While one batch of events is being collected, another may be being matched with subscriptions, and an older batch of notifications may be being distributed. The following subsections describe these phases of processing and the components that execute them.

Event Collection

Event providers are components that collect events and submit them to notification applications. Event providers can gather event data from the outside world proactively, or act as sinks to which external event sources push information. For example, an event provider could be a component that constantly polls an external data source, or it could be a web service that receives data passed to it by external callers.

An application can have several event providers, each potentially submitting events from a different event source. Event providers can be components that run within the SQL-NS engine, or they can be standalone programs that submit event data to your application. In the <Providers> element of the ADF, you declare which event providers your application will use and configure the options that control their operation.

SQL-NS provides several built-in event providers that can be used in an application without your having to write any code. You can also build your own event provider for your application that talks to a custom event source. Chapter 8, "Event Providers," provides details on all the built-in event providers, as well as the process of building a custom event provider for your application.

Listing 3.7 shows the <Providers> element from the stock application's ADF.

Listing 3.7. The Event Provider Configuration in the ADF

<Providers>
  <NonHostedProvider>
    <ProviderName>TestEventProvider</ProviderName>
  </NonHostedProvider>
</Providers>

In this application, rather than using a real event source, we're just going to submit some test events manually. Therefore, the ADF declares a single nonhosted event provider. The term "nonhosted" just means that the event provider is not hosted within the SQL-NS engine; the declaration in Listing 3.7 indicates that events will enter the application from an external program (in fact, we'll use a simple T-SQL script to submit events). Because there's only one event source in this application example, the event provider name is arbitrary. I've chosen the name TestEventProvider to signify that this is a placeholder used for testing. In a real application, which might have several real hosted and nonhosted event providers, event provider names can be useful to associate events with the providers that submitted them.

Note

Chapter 8 offers more details on both hosted and nonhosted event providers and describes when it's appropriate to use each type.

Subscription Management

Most notification applications provide users with a visual interface by which they can manipulate and manage their subscriptions. The form of this interface varies, depending on the nature of the pub-sub system. A stock quote application such as the one in this chapter might provide a website that a user can visit to enter a subscription. A line-of-business application might provide a way to subscribe for notifications in its standard Windows user interface.

Whatever the form of the external interface, subscription data must be ultimately transferred to the tables in the application's database. The implementation of the subscription management system can either insert rows into the subscription class views directly or use an API provided by SQL-NS to enter subscriptions. The subscription management API is covered in detail in Chapter 7.

There is no subscription management configuration in the ADF. The ADF contains declarations of subscription classes, but the systems by which subscriptions of those subscription classes are created and submitted to the application are treated as standalone entities that are not configured in the ADF. For the purposes of testing the application example in this chapter, we'll just enter subscription data directly into the views using a T-SQL script.

Generation

Generation is the phase during which events are matched with subscriptions to produce notifications. The generation component is provided by SQL-NS and uses either developer-defined or user-defined logic to determine whether a particular event matches a subscription.

SQL-NS exposes a variety of configuration options for controlling notification generation. Some of these options specify, for example, how and when matching logic is applied. These are configured in the ADF in the <Generator> and <ApplicationExecutionSettings> elements. Listing 3.8 shows the generator configuration for the stock application.

Listing 3.8. The Generator Configuration in the ADF

<Generator>
  <SystemName>%_NSServer_%</SystemName>
</Generator>
  ...
<ApplicationExecutionSettings>
  <QuantumDuration>PT15S</QuantumDuration>
</ApplicationExecutionSettings>

The <Generator> element specifies the system name, which tells the SQL-NS engine on which machine the generator should run. The <ApplicationExecutionSettings> element specifies a quantum duration, which controls how often the generator looks for new events to process. In this case, the quantum duration is set to 15 seconds. Chapters 11, "Debugging Notification Generation," and 12, "Performance Tuning," describe in more detail these and other options you can use to fine-tune generator operation.

Distribution

The distribution phase handles the formatting of notification data for a variety of delivery devices (email, cell phones, pagers, and so on) and the routing of the formatted notifications to delivery systems that gets them to their final destinations.

An application can have one or more distributors, possibly running on different servers. These are configured in the ADF's <Distributors> element. Listing 3.9 shows the <Distributors> element from the stock application's ADF.

Listing 3.9. The Distributor Configuration in the ADF

<Distributors>
  <Distributor>
    <SystemName>%_NSServer_%</SystemName>
    <QuantumDuration>PT15S</QuantumDuration>
  </Distributor>
</Distributors>

The stock application uses only one distributor, and this is declared in the single <Distributor> element. The configuration in Listing 3.9 specifies the distributor system name (the computer on which the distributor should run) and the distributor quantum duration (which controls how often the distributor polls for new batches of notifications to format and deliver15 seconds, in this case). Chapter 12 covers these and other distributor configuration options in greater detail.

Caution

If you are using the Standard Edition of SQL-NS, the event providers, generator, and distributor in your application must run on the same machine. This means that the <SystemName> element must have the same value in all the event provider, generator, and distributor declarations in your ADF. If you specify different system name values with SQL-NS Standard Edition, you get an error when compiling your application.

If you are using the Enterprise Edition of SQL-NS, you may configure the various components to run on different machines by specifying different values for the <SystemName> elements. This allows your application to scale out for better performance. Chapter 13, "Deploying a SQL-NS Instance," covers the additional configuration steps (beyond specifying system names in the ADF) required to enable a scale-out deployment.


Specifying Other Parts of the Stock Application

The ADF defines the core of the application, but several other pieces are required to make it run. This section briefly describes those pieces as they pertain to the stock application.

The Instance

SQL-NS has the concept of instances, in much the same way that SQL Server does. A SQL-NS instance is a single, named configuration of SQL-NS that can host one or more applications. Each instance is an independent entity that can be started, stopped, and configured on its own. In SQL-NS, an instance is defined by an Instance Configuration File (ICF). The ICF defines the list of applications in the instance (here, just onethe StockWatcher application) and the set of delivery channels that applications in the instance can use to send notifications.

This sample's ICF creates a Notification Services instance called StockBroker. I've included the text of the ICF in Listing 3.10, but I will not go into an explanation of it here because most of the details are not relevant at this point. Chapter 4, "Working with SQL-NS Instances," covers the instance concepts and all the elements of the ICF in detail. If you have the Management Studio project for this sample open, you'll find the ICF under the Miscellaneous node in the StockBroker projectthe file name is InstanceConfiguration.xml.

Listing 3.10. The ICF That Defines the SQL-NS Instance That Hosts the Stock Application

[View full width]
<?xml version="1.0" encoding="utf-8" ?>
<NotificationServicesInstance xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:xsi=http:/
  /www.w3.org/2001/XMLSchema-instance xmlns="http://www.microsoft.com
  /MicrosoftNotificationServices/ConfigurationFileSchema">

  <InstanceName>StockBroker</InstanceName>

  <ParameterDefaults>
    <Parameter>
      <Name>_SQLServer_</Name>
      <Value>... Your SQL Server Name ...</Value>
    </Parameter>
    <Parameter>
      <Name>_NSServer_</Name>
      <Value>%COMPUTERNAME%</Value>
    </Parameter>
    <Parameter>
      <Name>_InstanceBaseDirectoryPath_</Name>
      <Value>C:\SQL-NS\Samples\StockBroker</Value>
    </Parameter>
  </ParameterDefaults>

  <SqlServerSystem>%_SQLServer_%</SqlServerSystem>

  <Database>
    <DatabaseName>StockBroker</DatabaseName>
    <SchemaName>NSInstance</SchemaName>
  </Database>

  <Applications>
    <Application>
      <ApplicationName>StockWatcher</ApplicationName>
      <BaseDirectoryPath>%_InstanceBaseDirectoryPath_%\StockWatcher</BaseDirectoryPath>
      <ApplicationDefinitionFilePath>ApplicationDefinition.xml</ApplicationDefinitionFilePath>
      <Parameters>
      <Parameter>
        <Name>_NSServer_</Name>
        <Value>%_NSServer_%</Value>
      </Parameter>
      <Parameter>
        <Name>_ApplicationBaseDirectoryPath_</Name>
        <Value>%_InstanceBaseDirectoryPath_%\StockWatcher</Value>
      </Parameter>
      </Parameters>
    </Application>
  </Applications>

  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      <ProtocolName>File</ProtocolName>
      <Arguments>
        <Argument>
          <Name>FileName</Name>
          <Value>%_InstanceBaseDirectoryPath_%\FileNotifications.txt</Value>
        </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>

</NotificationServicesInstance>

As it does with the ADF, the SQL-NS compiler creates a set of database objects for the instance from the information provided in the ICF. The <Database> element in the ICF specifies where these objects go. In this case, you can see that they are deployed into the same StockBroker database as the application objects, but isolated in a different schema called NSInstance.

Notice that the ICF contains a reference to the ADF for each declared application. In Listing 3.10, the <Application> element that declares the StockWatcher application includes an <ApplicationDefinitionFilePath> element that provides the path to the ADF. At compile time, you pass just the ICF to the SQL-NS compiler. Based on the <ApplicationDefinitionFilePath> elements in the ICF, the compiler locates, loads, and compiles each application's ADF.

Entering Event and Subscription Data

If this were a real application, it would use an event provider that would read and submit real stock data from one of many stock market data publishers. It would also have a subscription management interface by which users could enter subscriptions. Because the focus of this chapter is the core SQL-NS concepts, not the mechanics of reading from stock market data sources or implementing user interfaces for subscription management, we'll use a simple test script to submit event and subscription data. The data submitted through this script will allow us to observe how the application and the SQL-NS platform work. Chapters 7 and 8 cover the building of subscription management interfaces and event providers in detail.

The test script that submits event and subscription data is included in the StockBroker Management Studio project. To open the script, expand the Queries node under the StockBroker project in Solution Explorer and select the Test.sql file. Do not run this file yetthe database objects it relies on don't exist yet because we haven't compiled the SQL-NS ICF and ADF. Just open the file to look at its contents for now.

The script first creates subscriber and subscriber device records for four test subscribers. The records are created by inserting rows into subscriber and subscriber device views in the instance schema in the StockBroker database. It then enters a subscription for each subscriber by inserting rows into the subscription class view in the StockWatcher application schema. Finally, it inserts a set of test events into the event class view. After compiling the application, we'll run this script to test it.

Seeing the Final Notifications

After raw notification data is generated, it gets formatted and then delivered. In this sample, we'll use the built-in XsltFormatter content formatter provided by SQL-NS that turns the raw notification into a readable message using an XSL transform. Although the XSLT formatter is used in the examples in the next several chapters, I'll defer explaining the details of how it works until Chapter 9, "Content Formatters."

On the delivery side, this sample application uses another SQL-NS built-in component: the File Delivery Protocol. This component "delivers" the notifications to a file. You can then check the output of the application simply by opening this file. The subscriber device records created by the test script you examined in the previous section tell the SQL-NS distributor to route the formatted notifications for the test subscribers to the output file. In a real application, you'd probably use several delivery protocols that could send the notification messages via email or as text messages to cell phones, for example. Chapter 10 describes all the built-in SQL-NS delivery protocols and shows how you can build your own custom delivery protocol.


Running the Stock Application

In this section, we'll compile, register, and enable the application; start its engine running; and then feed it sample data. We'll submit events and subscriptions as described earlier and then observe the notification data written to the output file.

Assuming that your development environment is set up as described in Chapter 2, you can complete the steps described here to get the application going. In this chapter, I've just listed the steps without explaining the purpose of each one in detail. Chapter 4 goes into the details of what these steps actually do.

1. | If you have not already done so, open the C:\SQL-NS\Samples\StockBroker\StockBroker.ssmssln solution in Management Studio.
2. | If the Object Explorer pane is not visible, open it by selecting Object Explorer from the View menu, or by pressing F8.
3. | In the Object Explorer, connect to the SQL Server you plan to use for SQL-NS development. To do this, click the Connect button, choose Database Engine from the drop-down list, enter the connection information in the dialog box that appears, and finally, click Connect. If you're using SQL Server Authentication, make sure you connect using the credentials of your development account.
4. | In the Object Explorer tree, find the Notification Services folder. It should be empty, unless any SQL-NS instances already exist on the SQL Server.
5. | Right-click the Notification Services folder and select New Notification Services Instance.
6. | In the New Notification Services Instance dialog box, click the Browse button to browse for the stock instance's ICF. In the File Open dialog box, navigate to C:\SQL-NS\Samples\StockBroker and select the InstanceConfiguration.xml file. Caution If the C:\SQL-NS\Samples\StockBroker\InstanceConfiguration.xml file appears to be missing on your system, you probably have not completed all the setup steps given in Chapter 2. If this is the case, return to Chapter 2 and make sure you complete the steps in the "Customizing the Source Files for Your Environment" section (p. 39).
7. | The Parameters grid in the New Notification Services Instance dialog box will be populated with the parameters from the ICF. You should not have to change any of these values.
8. | Check the box labeled Enable Instance After It Is Created, below the Parameters grid.
9. | Click the OK button to begin compiling the instance and its associated application. A progress dialog box will appear, listing the various steps being performed. When all steps are complete, you should see the status reported as Success. When this happens, click Close to dismiss the progress dialog box.
10. | Now you should see a new item in the Notification Services folder in the Object Explorer tree: an instance named StockBroker.
11. | Right-click this new instance and select Tasks, and then select Register from the context menus. The Register Instance dialog box opens.
12. | Check the box labeled Create Windows Service.
13. | Beneath the Service Logon label, enter the name and password for the Windows user account you created in Chapter 2 to run the SQL-NS service. If your service user account is a domain account, don't forget to qualify the account name with the domain name.
14. | If you're using SQL Server Authentication, select the option button labeled SQL Server Authentication beneath the Authentication label. Enter the SQL login name and password for the test account you created in Chapter 2. If you're using Windows Authentication, leave the Windows Authentication option button selected.
15. | Click OK to begin registering the instance. A progress dialog box listing the various registration operations will appear. When registration completes successfully, click the Close button to dismiss the progress dialog box.
16. | In the Solution Explorer, expand the Queries folder under the StockBroker project and open the SQLPermissions.sql file. Execute the code in this file by pressing F5. This grants the SQL-NS service permission to access the application's database.
17. | Right-click the StockBroker instance in the Object Explorer and select Start from the context menu. Click Yes in the confirmation dialog box to proceed with starting the SQL-NS service for the instance.

At this point, the application should be running, and we can start submitting subscriptions and events. If your service does not start correctly, refer to Chapter 15, "Troubleshooting SQL-NS Applications," for suggestions on how to diagnose the problem.

To add subscribers, subscriptions, and events, we'll use the T-SQL test script you examined earlier:

1. | Open the Test.sql file in Management Studio. The file is located under the Queries folder in the StockBroker project.
2. | Before you run Test.sql, open the FileNotifications.txt file from the Miscellaneous folder under the StockBroker project in the Solution Explorer. This is the output file to which the notifications will be written when events are matched against subscriptions. When you first open it, the file should be blank.
3. | Go back to Test.sql and execute its contents by pressing F5. Caution If the Test.sql script reports error messages saying that it failed to add subscribers because subscription management is disabled for the instance, you probably forgot to check the Enable Instance After It Is Created check box in step 8 of the earlier instructions. If you see these error messages, right-click the StockBroker instance in the Object Explorer and select Enable from the context menu. Click Yes in the confirmation dialog box and then rerun Test.sql.

After Test.sql completes, the SQL-NS engine begins processing the events and subscriptions by executing the matching logic query defined in the ADF. The distributor in the SQL-NS engine picks up the rows of data that result from the matching query, formats them, and then writes the formatted messages out to the notifications file. Within about a minute, Management Studio should pop up a dialog box telling you that the FileNotifications.txt file has changed and asking you if you want to reload it. Click Yes to reload the file and you will see the contents of the notifications that were generated.

The particular event and subscription data submitted by the Test.sql script was chosen to generate some matches. If you go back and look at the data values in Test.sql, you'll easily see which events and subscriptions satisfy the matching criteria defined by the query in the ADF. The contents of the FileNotifications.txt file should reflect these matches.

Feel free to edit Test.sql and enter some new events and subscriptions with different values. If you do this, be certain that you don't rerun the statements that enter subscriber and subscriber device records: SQL-NS does not allow duplicate subscribers or subscriber devices, so these statements should be run only once. After you submit new events and subscriptions, any matches that occur will result in new notification messages written to FileNotifications.txt.


Inside the Running Application

When the SQL-NS compiler processed the ADF for the stock application, it created database objects from the information in it. When we registered the instance, SQL-NS installed a Windows Service to host its execution engine. These two pieces, the database objects and the Windows Service, are the key components of the running application. This section examines both to illustrate how the application really works.

The Database

Connect the Object Explorer in Management Studio to your SQL Server and expand the Databases node. You should see a new database called StockBroker (recall that this is the database name specified in the <Database> elements in the ICF and ADF). The SQL-NS compiler created this database when it compiled the ICF and placed database objects associated with the instance and application in it.

The instance database objects are created in a schema called NSInstance (this was the name specified in the <SchemaName> element in the ICF). If you expand the StockBroker database and look at the list of tables, views, and stored procedures, you'll see several objects prefaced with the NSInstance schema name. For now, we'll ignore these instance database objects and focus on the application database objects instead. We'll revisit the instance database objects in Chapter 4.

In the list of tables, views, and stored procedures in the StockBroker database, many objects are qualified with the StockWatcher schema name. These are all the application database objects the compiler created for our StockWatcher application. Many of these are for internal use by the SQL-NS engine, so we won't explore them here. But I do want to show you a few of them to give you an idea of what the compiler did with the information in the ADF.

Look for a table called StockWatcher.NSStockPriceChangeEvents. If you look at the columns in this table, you'll see that all the fields from the StockPriceChange event class declaration have become columns in this table. SQL-NS has also added its own extra columns that it uses for internal tracking. This table is used to store events and was created by the compiler from the event class definition. Similarly, you'll see a table called StockWatcher.NSStockPriceHitsTargetSubscriptions that matches the schema of the StockPriceHitsTarget subscription class and is used to store subscriptions of that class. As you might expect, there is also a StockWatcher.NSStockAlertNotifications table that stores StockAlert notification data.

The Views node under the database in the Object Explorer shows the list of views the compiler created. In this list, you'll find all the views used to insert data in the Test.sql script we ran earlier.

Also, look at the StockWatcher.NSFireDeveloperDefined1 stored procedure. In Object Explorer, stored procedures are located under the Programmability node beneath the database, in the Stored Procedures subnode. To see the definition of the StockWatcher.NSFireDeveloperDefined1 stored procedure, right-click it, select Script Stored Procedure As, CREATE To, and then New Query Editor Window in the series of context menus. As the definition shows, StockWatcher.NSFireDeveloperDefined1 wraps the text of the <Action> element in the match rule declaration in the ADF. Look for a comment that says /** APPLICATION DEFINED RULE STARTS HERE **/. Immediately after this comment, you'll see the rule text we supplied in the ADF. When the SQL-NS engine needs to invoke this matching logic, it executes this stored procedure.

Spend some time looking at other objects in the database to get a feel for what's there. Don't be alarmed by the number of objects you seeyou'll only ever have to deal with a small number of these as you build SQL-NS applications. The rest are there to support the rich features that SQL-NS provides.

The Windows Service

In addition to the tables and stored procedures just examined, the SQL-NS compiler also stores the component configuration information from the ADF in the database. On startup, the Windows service connects to the database and reads this configuration. This includes the number and type of event providers, as well as the configuration of the generator and distributors. Using this configuration information, the service instantiates the appropriate execution components and starts them running.

Our application had a single generator, configured with a quantum duration of 15 seconds. On startup, the Windows service instantiated a generator component that checks the database for new batches of events every 15 seconds. When our test script wrote events to the database, the generator processed them by executing the stored procedures that contain our rule logic. This resulted in notification data stored in the notification tables.

The single distributor we configured in our ADF was also represented by a running component in the Windows service, instantiated on startup. This component periodically (again, every 15 seconds, as specified by the distributor's quantum duration) checks the database for new notifications to format and deliver. When the evaluation of the matching logic by the generator caused the notifications to be written to the notifications table, the distributor read them, passed them through the content formatter we specified, and then wrote them to the output file using the file delivery protocol.

Figure 3.7 shows the Windows service and its interactions with the database.

Figure 3.7. The running Windows service interacting with the database.

[View full size image]


What Has the SQL-NS Platform Provided?

In its final form, the notification application is a database application coordinated by a Windows service. To build the application, we had to write a relatively small amount of code. In fact, all we provided were the schemas for events, subscriptions, and notifications; the matching logic; and component configuration. From this, SQL-NS constructed the database application, including the tables, views, and stored procedures.

In the previous section, we examined the parts of the database that resulted from the information specified in the ADF. We did not look into the multitude of other supporting tables, views, stored procedures, and functions required by the running application. These include code that maintains state about the application and enables it to recover gracefully after an unplanned shutdown, support for garbage collection of old event and notification data that has been completely processed, and several reporting and debugging utilities that can provide insight into what the running application is actually doing. All these were created for us, and we did not have to understand how they are implemented.

Many other benefits of building this application on SQL-NS have not been visible because we've run the application on such a small scale. But in fact, the application as it stands will support large volumes of events and subscriptions. (The exact number depends on the hardware you're using, but even on a modest developer system, it's possible to run applications with tens of thousands of events and subscriptions.)

Apart from the supported data volume, another aspect of the application that has not been visible because of the limited way in which we've used it is its manageability and reporting capabilities. For example, using simple SQL-NS tools, we could disable certain components of the application while leaving others running. This would allow us to perform incremental hardware and software maintenance without a total shutdown of the system. We could also obtain reports about the exact numbers of events and subscriptions going through the system. Chapter 14, "Administering a SQL-NS Instance," examines how to do these tasks, but even in this simple application, these capabilities are present because of what the SQL-NS platform has provided.

Furthermore, the application can be maintained and extended easily. If we wanted to add a new type of event or a new subscription class, for instance, all we would have to do is add the necessary schema and logic to the ADF and then run SQL-NS tools to update the application (Chapter 5, "Designing and Prototyping an Application," introduces the tools and techniques for doing this). Updating the application would automatically modify existing database entities and create the new ones necessary to support the extensions. If we were building the application from scratch, this type of incremental update would likely be much more difficult.

If you were building the application from scratch, you would also have to do vast amounts of testing to ensure the reliability and security of the execution engine. Using SQL-NS, you can be sure that the execution engine has been designed with reliability and security in mind and has already gone through rigorous testing in these areas.

In summary, although building just the core functionality of the application from scratch might not take that much longer than it does with SQL-NS, all the supporting features that make the application truly rich would be time consuming to build and difficult to get just right. SQL-NS provides these capabilities for free. Using SQL-NS reduces the time to market for your application and increases its reliability, security, and scalability.

Extending the Stock Application

Although the core of the application built here looks much like it would if this were a real application, its inputs and outputs do not. We have been submitting events and subscriptions from hard-coded test files and looking at notification output through text files.

To make this application real, we would have to add a real event provider that read from a real event source. We would build a full subscription management interface to allow users to manage their subscriptions, and we would add support for delivery to real devices. Later chapters show how to do all these tasks.

Apart from input and output pieces, there is work we would do on the core of the application to make it more real. One limitation of the way the application works today is that it stores no state. Therefore, subscribers get notified of every stock price change that is above the price target value they specified in their subscription. If you had a subscription for XYZ stock with a price target of $50, you'd get notified when the stock hit $51, then again if it fluctuated to $51.25, again if it went back down to $51, and so on. For any user, this flood of notifications would quickly become annoying.

To deal with this, the application could store state information about the last price for which each subscriber was notified for a given stock symbol. It could then use this data to filter out price change events that are just intermediate fluctuations. Maintaining state is a core part of the SQL-NS application model, and Chapter 6 covers this in more detail.

Also, all the subscriptions in this chapter's application are event triggered. That is, they fire in response to an incoming event. We could also support scheduled subscriptions: subscriptions that fire at a particular time of day. This would allow you, for example, to enter a subscription that said, "Notify me of the price of stock XYZ at 5:00 p.m. every weekday."

This type of subscription does not fire in response to any given stock price change event. Rather, as stock price events come in continuously during the day, the application would have to keep track of the latest price for XYZ. At 5:00 p.m., it would send you a notification message containing the latest price. This type of subscription is also supported directly in the SQL-NS application model, and in Chapter 6, you'll see how to implement it.


Cleaning Up the Instance and Application

When you're finished examining the instance and application created in this chapter, you can use the following steps to remove them from your system. These steps clean up the databases, Registry keys, and other settings associated with the instance and application.

Note

In all the following instructions, when you are instructed to right-click the instance in the Object Explorer, make sure you right-click the StockBroker instance in the Notification Services folder, not the StockBroker database in the Databases folder.

1. | Stop the SQL-NS service by right-clicking the instance in the Notification Services folder in Management Studio's Object Explorer, selecting Stop from the context menu, and then clicking Yes in the confirmation dialog box. Upon success, dismiss the progress dialog by clicking the Close button.
2. | Right-click the StockBroker instance and select Tasks, and then Unregister. Confirm the unregister operation by clicking Yes in the confirmation dialog box. Upon success, dismiss the progress dialog by clicking the Close button.
3. | Right-click the StockBroker instance and select Tasks, and then Delete. Confirm the delete operation by clicking Yes in the confirmation dialog box.


Summary

In the SQL-NS application model, both events and subscriptions are just data. SQL-NS uses SQL joins to match the events and subscriptions, and because SQL Server can usually execute these efficiently, notification applications built to this model tend to scale well.

Building an application on SQL-NS involves writing an ADF: an XML document that describes the schemas, logic, and component configuration of the application. This chapter described the ADF for a sample application: one that sends stock price updates to subscribers. The information in this chapter should give you a feel for SQL-NS applications and what's involved in building them. Later chapters provide the details and teach you how to build your own applications.


Chapter 4. Working with SQL-NS Instances

IN THIS CHAPTER

  • What Is an Instance?

  • Creating an Instance and Making It Run

  • Examining the Parts of the Instance

  • Deployment of SQL-NS Instances on Multiple Servers

  • The Instance Configuration File in Detail

  • Cleaning Up the Instance

This chapter examines SQL Notification Services instances: the fundamental units of isolation, deployment, and administration in SQL-NS. Every SQL-NS application you build will be part of some instance, so it's important to understand what an instance is and how it operates. This chapter covers the basic concepts behind instances and describes how to create one. Chapter 13, "Deploying a SQL-NS Instance," and Chapter 14, "Administering a SQL-NS Instance," describe instance operations, security, and deployment configuration.

Note

Before we begin, a word of warning: This chapter teaches a number of mechanical details and manual steps that might seem laborious. Don't be discouraged by this: Most of what you learn in this chapter you'll only do manually once. After you get through and understand this material, you'll likely automate all these steps and never have to think about them again. But, it's important to go through and understand them once so that you know what's going on behind the scenes. It's important to know what each step is doing and why it's necessary to do it; it's not important, however, that you memorize the exact syntax of each command or the precise sequence of steps.


What Is an Instance?

When you install SQL-NS, you get a set of tools and supporting files placed on your computer, along with the Registry settings that the installer and uninstaller need. However, the installer does not create any SQL-NS instances. You must use the SQL-NS tools to create instances when you need them, to host your various SQL-NS applications.

An instance is a named configuration of SQL-NS that contains one or more SQL-NS applications. Think of an instance as a container that groups a set of applications with common security and administration settings. Subscriber information is shared among applications within an instance: When a new subscriber registers (using a subscription management interface such as the one we'll build in Chapter 7, "The SQL-NS Subscription Management API"), the registration information is stored at the instance level and made available to all applications in the instance.

Several parts make up the physical representation of an instance:

  • Database objects encapsulating the instance data and logic

  • Database objects for each application in the instance

  • Metadata in system tables that describes properties of the instance

  • Registry keys that store parts of the instance configuration

  • A dedicated SQL-NS engine that hosts the runtime components of the applications in the instance

  • Performance counters used to monitor the instance

These are shown in Figure 4.1. In this chapter, we'll create a simple instance and then examine each of these parts in detail.

Figure 4.1. Parts of a SQL-NS instance.

[View full size image]


Creating an Instance and Making It Run

To create an instance, you write an Instance Configuration File (ICF) and compile it with the SQL-NS compiler. An ICF is an XML document that describes the instance and the applications in it. The ICF contains a reference to the ADF for each application.

After you've written your ICF, there are usually five steps to getting the instance running:

1. | Compile the ICF using the SQL-NS compiler.
2. | Register the instance.
3. | Grant appropriate permissions for database and filesystem access.
4. | Enable the parts of the instance you want to run.
5. | Start the SQL-NS engine.

Step 1 creates the instance and application database objects and populates them with the required data. It also installs the required metadata about the instance in the SQL-NS system tables.

Step 2 installs the Registry keys, the Windows service that hosts the SQL-NS engine, and the performance counters. A separate Windows service is installed for each SQL-NS instance. This allows different instances to be managed and configured independently.

Step 3 gives the SQL-NS engine permission to log in to the database and perform the data operations required to run the instance. The instance and application database objects are created by an administrator (the user who invokes the SQL-NS compiler in step 1) and, by default, only that user will have access to them. Because it's recommended that the SQL-NS engine run as some low-privileged (nonadministrator account), you have to manually grant it login and database access. In some cases, the components in the SQL-NS engine also need to read from and write to files in the filesystem. Again, because the engine will be running as a low-privileged account, you'll need to explicitly grant it access to the necessary files and directories.

Step 4 is required because when an instance is first created, all applications and components within it are configured as disabled. SQL-NS configures things this way for security reasons. Because you have to explicitly enable the things you want to run, the chances of you accidentally running something that poses a security risk are reduced.

Step 5 puts the instance in motion. If the SQL-NS engine is hosted in a Windows service (as it usually is), you can start the service from the command line (using the net start command), from Management Studio, from the SQL Server Configuration Manager, or from the Services applet in Administrative Tools. If you have the SQL-NS engine hosted in another application (as described in Chapter 17, "Hosting the SQL-NS Execution Engine") you'll start the engine via whatever controls the hosting application offers.

For the purpose of illustration in this chapter, I've provided a simple ICF that defines a skeleton instance with two applications in it. This ICF and the associated ADFs contain the minimum amount of code required to create a multiapplication instance that actually compiles and runs. A real instance with useful applications will likely have more code than you'll see here, but to highlight the basic instance concepts, the simple files referenced in this chapter will be sufficient. In the following sections, we'll work through the steps to get the instance running and examine the various parts of the instance in detail.

Note

To work through the steps in this chapter, you need to set up your development environment as described in Chapter 2, "Getting Set Up." If you have not already done so, perform the steps in that chapter before proceeding.

The source code for this chapter is located in the Samples\MinimalMultiApplication directory under the source code base directory.

Tools for Working with SQL-NS Instances

SQL-NS ships with two sets of tools for working with instances:

  • Command-line tools you can invoke from a command prompt

  • Graphical user-interface tools in Management Studio

The two sets of tools offer equivalent functionality. The command-line tools are easier to automate through scripts, but the graphical tools are more user friendly at first.

In Chapter 3, "The Simplest Notification Application: Stock Quotes," you used the graphical tools to create and run the StockBroker instance, but I didn't explain the tools in much detail. In this chapter, we'll take a more detailed look at what the tools do. I'll explain how to perform the various steps outlined in the previous section using both the command-line and the graphical tools.

Caution

As you read through this chapter, you should read the instructions for both toolsets in each section so that you become familiar with all the tools available to you. However, you should only execute the instructions for one of the two toolsets when you perform the various SQL-NS operations on your system.

Because the two sets of tools do essentially the same things, if you complete the instructions for both, you'll be carrying out the underlying SQL-NS operations twice. In many cases, this will lead to errors or unexpected behavior.

After you become familiar with SQL-NS concepts, I think you'll find the command-line tools most convenient to use. After this chapter, all the instructions for working with the examples will refer only to the command-line tools (though we'll still use Management Studio to examine and edit ICFs, ADFs, and T-SQL scripts). However, based on what you learn in this chapter, you should be able to perform the equivalent steps using the graphical tools if you prefer.

The following subsections highlight the key points about each set of tools. The instructions in the remainder of this chapter provide more details on how they're used to carry out various operations.

Note

The functionality of the SQL-NS tools is also exposed programmatically through the Notification Services Management Objects (NMO) API. Using this API, you can write code to invoke the same operations that the tools usually perform. The NMO API is covered in Chapter 16, "Using Notification Services Management Objects."

SQL-NS Command-Line Tools

The SQL-NS command-line tools are invoked through an executable called nscontrol.exe. This executable is installed in the SQL-NS binaries directory. If you installed SQL Server 2005 in the C:\Program Files\Microsoft SQL Server directory, the SQL-NS binaries, including nscontrol.exe, will be in C:\Program Files\Microsoft SQL Server\90\NotificationServices\9.0.242\bin.

To work with nscontrol.exe, it's convenient to have the SQL-NS binaries directory in your path. I recommend that you open a Notification Services command prompt when you want to work with SQL-NS command-line toolsyou can access the Notification Services Command Prompt shortcut via the Start menu, All Programs, Microsoft SQL Server 2005, Configuration Tools, Notification Services Command Prompt. This shortcut opens up a command prompt window with the path already configured to include the SQL-NS binaries directory. The instructions for using the command-line tools in this chapter assume you're invoking the tools from a Notification Services Command Prompt.

The command-line arguments passed to nscontrol.exe determine what operations it performs. Usually, the first argument is a command name: for example, the command name, create, is given to instruct nscontrol.exe to create a new SQL-NS instance by compiling an ICF. Each command requires a specific set of parameters, which are passed as additional arguments to nscontrol.exe. In later sections, when nscontrol.exe is used to carry out various SQL-NS operations, the different command names and expected arguments are explained.

SQL-NS Graphical Tools in Management Studio

The SQL-NS tools built in to Management Studio are all invoked from the Object Explorer. When you connect Management Studio's Object Explorer to a Standard or Enterprise Edition SQL Server, you'll see a folder labeled Notification Services in the object tree for that server. The items in this folder represent the various SQL-NS instances that have databases on the SQL Server. If there are no SQL-NS instances on the SQL Server, the Notification Services folder is empty.

All the SQL-NS commands are available from context menus that appear when you right-click the Notification Services folder or its contents. When you right-click the folder itself, you get a context menu containing commands that apply to the server as a whole. When you right-click a particular instance in the Notification Services folder, the context menu contains commands that apply to that individual instance. Many of the commands launched from the context menu bring up dialog boxes that ask for additional information. As you work through the steps in this chapter, you'll become familiar with each of these commands and their associated dialog boxes.

Compiling the ICF

Listing 4.1 shows the complete ICF for this chapter. Glance at it to get a feel for what it contains, but don't be concerned if you don't understand every line of it right now. First, we'll focus on compiling and registering the instance so that we can examine the databases and Registry keys that get created. Doing this illustrates many of the important instance concepts you should understand before we delve into the code. I will explain the code in the ICF line-by-line in the section "The Instance Configuration File in Detail" (p. 103).

Listing 4.1. The Instance Configuration File

[View full width]
<?xml version="1.0" encoding="utf-8"?>
<NotificationServicesInstance xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:xsi=http:/
  /www.w3.org/2001/XMLSchema-instance xmlns="http://www.microsoft.com
  /MicrosoftNotificationServices/ConfigurationFileSchema">

  <InstanceName>MinimalMultiApplication</InstanceName>

  <ParameterDefaults>
    <Parameter>
      <Name>_SQLServer_</Name>
      <Value>... Your SQL Server Name ...</Value>
    </Parameter>
    <Parameter>
      <Name>_NSServer_</Name>
      <Value>%COMPUTERNAME%</Value>
    </Parameter>
    <Parameter>
      <Name>_InstanceBaseDirectoryPath_</Name>
      <Value>C:\SQL-NS\Samples\MinimalMultiApplication</Value>
    </Parameter>
  </ParameterDefaults>

  <SqlServerSystem>%_SQLServer_%</SqlServerSystem>

  <Database>
    <DatabaseName>MinimalMultiApplication</DatabaseName>
    <SchemaName>NSInstance</SchemaName>
  </Database>
  <Applications>
    <Application>
      <ApplicationName>Application1</ApplicationName>
      <BaseDirectoryPath>
        %_InstanceBaseDirectoryPath_%\Application1
      </BaseDirectoryPath>
      <ApplicationDefinitionFilePath>
        ApplicationDefinition.xml
      </ApplicationDefinitionFilePath>
      <Parameters>
        <Parameter>
          <Name>_NSServer_</Name>
          <Value>%_NSServer_%</Value>
        </Parameter>
        <Parameter>
          <Name>_ApplicationBaseDirectoryPath_</Name>
          <Value>%_InstanceBaseDirectoryPath_%\Application1</Value>
        </Parameter>
      </Parameters>
    </Application>
    <Application>
      <ApplicationName>Application2</ApplicationName>
      <BaseDirectoryPath>
        %_InstanceBaseDirectoryPath_%\Application2
      </BaseDirectoryPath>
      <ApplicationDefinitionFilePath>
        ApplicationDefinition.xml
      </ApplicationDefinitionFilePath>
      <Parameters>
        <Parameter>
          <Name>_NSServer_</Name>
          <Value>%_NSServer_%</Value>
        </Parameter>
        <Parameter>
          <Name>_ApplicationBaseDirectoryPath_</Name>
          <Value>%_InstanceBaseDirectoryPath_%\Application2</Value>
        </Parameter>
      </Parameters>
    </Application>
  </Applications>

  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      <ProtocolName>File</ProtocolName>
      <Arguments>
        <Argument>
          <Name>FileName</Name>
          <Value>
            %_InstanceBaseDirectoryPath_%\FileNotifications.txt
          </Value>
        </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>

</NotificationServicesInstance>

Compiling the ICF with the Command-Line Tools

To compile the ICF with the SQL-NS command-line tools, you use the nscontrol create command. This command takes the name of the ICF as a parameter. It reads the XML in the ICF and the referenced ADFs and uses it to create the instance and application database objects.

I've provided a batch file that invokes nscontrol create with the ICF shown in Listing 4.1. The batch file is useful because the actual call to nscontrol create is fairly long. Retyping it each time we needed to run it would not only be tiresome, it would also be error prone.

Tip

It's good practice to use batch files to invoke nscontrol commands. Many commands take several parameters, and using a batch file means that you don't have to remember all these parameters every time.

Take a look at the call to nscontrol create in this batch file to examine how it works.

Open the create.cmd file, located in the Samples\MinimalMultiApplication\Scripts subdirectory off your source code base directory, in a text editor. You'll see some lines at the top that have to do with setting environment variables and getting the username and password required for SQL Server Authentication: You can ignore these for now. But, below that, the actual call to nscontrol create appears as follows:

[View full width]

nscontrol create -in "%INSTANCE_BASE_DIRECTORY_PATH%\InstanceConfiguration.xml"
  %SQL_USER_INFO% _SqlServer_=%SQL_SERVER% _NSServer_=%NS_SERVER%
   _InstanceBaseDirectoryPath_= "%INSTANCE_BASE_DIRECTORY_PATH%"

The -in argument specifies the name of the ICF to compile. In this case, it's InstanceConfiguration.xml located in the base directory of the instance (the name of which is obtained from the %INSTANCE_BASE_DIRECTORY_PATH% environment variable, established in one of the other scripts used to set up the development environment).

Next, the value of the environment variable, %SQL_USER_INFO%, is given. The value of this environment variable is set by one of the setup scripts invoked earlier in create.cmd. If you're using SQL Server Authentication, this environment variable is set to a string that supplies values for the -sqlusername and -sqlpassword arguments to nscontrol create. These arguments tell nscontrol create what username and password to use when connecting to the SQL Server; the values supplied are the username and password of the development account you set up in Chapter 2. If you're using SQL Server Authentication, the expanded value of %SQL_USER_INFO% might look like

-sqlusername "SQL-NS_Dev" -sqlpassword "devPassword"

If you're set up to use Windows Authentication, %SQL_USER_INFO% is set to an empty string. This results in the -sqlusername and -sqlpassword arguments being omitted, which causes nscontrol create to use Windows Authentication.

The remaining arguments passed to nscontrol create specify values for parameters used in the ICF. For example, the argument _SqlServer_=%SQL_SERVER% instructs the compiler to replace every occurrence of the string %_SqlServer_% in the ICF with the value of the %SQL_SERVER% environment variable.

The last two arguments in the nscontrol create command line specify values for parameters _NSServer_ and _InstanceBaseDirectoryPath_. When we go through the code line-by-line in the section "The Instance Configuration File in Detail" (p. 103), you'll see where these parameters are used. Parameters are just a convenient text substitution mechanism that makes it easy to apply certain values consistently throughout your ICF and ADF code. The number and names of parameters used in the ICF are not mandated by SQL-NS. You may use parameters as much or as little as you prefer, and you may give your parameters any names you choose. I always surround the words in my parameter names with underscores, but that's just a convention that makes it easy to distinguish parameters from other tokens in the file. For more information on parameters, see the "Parameters" section (p. 104).

Tip

At the minimum, it's good practice to use parameters for server names in the ICF that may change. This makes it easy to relocate your instance to a different server without having to modify your ICF code directly.

Also, whenever you need to include sensitive information in the ICF, such as usernames and passwords, it's usually best to use parameters. (Usernames and passwords are sometimes needed for event providers and delivery protocols, as described in Chapter 8, "Event Providers," and Chapter 10, "Delivery Protocols.") By using parameters, you can avoid storing these sensitive strings in the file, where they may be visible to unauthorized users.

In the example we just examined, all the parameter values came from environment variables. You can also specify parameter values that are literal constantsfor example, _SqlServer_=DBServer01. However, using environment variables is usually more convenient. The development environment was set up in Chapter 2 so that all the environment variable values are set in one batch file and then used in all the other batch files that invoke nscontrol commands. With this setup, if you need to change a value, you just need to make the change in one place.

Now that we've looked at the nscontrol create command, let's run it and see what it does:

1.
Open a Notification Services Command Prompt on your development machine and navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MinimalMultiApplication\Scripts

2.
Run create.cmd. If you're using SQL Server Authentication, the script prompts you for the SQL password for the development account you created in Chapter 2. If you're using Windows Authentication, you will not be prompted.

The script should generate output similar to that shown in Figure 4.2, indicating that SQL-NS is validating the instance definition and creating the database objects for the instance and the two applications within it.

Figure 4.2. Running nscontrol create on the command line.

[View full size image]

Compiling the ICF with the Management Studio Tools

To compile the ICF with the graphical SQL-NS tools in Management Studio, you use the New Notification Services Instance command. This command is launched from the context menu opened by right-clicking the Notification Services folder in the Object Explorer tree.

When you invoke this command, Management Studio opens the New Notification Services Instance dialog box, in which you select the ICF to compile. After you select an ICF, Management Studio populates the parameters grid in the New Notification Services Instance dialog box with the list of parameters used in the file. The values in the grid are initialized with any defaults you may have supplied in the ICF (as described later in the "Parameters" section, p. 104). If you want to change any of the parameter values, you can do so by editing the values shown in the grid. The New Notification Services Instance dialog box also has a check box control by which you can choose whether to enable the instance after it has been created (enabling of instances is covered in more detail in the "Enabling the Instance" section, p. 95, later in this chapter).

Follow these instructions to compile this chapter's sample instance with the Management Studio tools:

1. | Open Management Studio and connect the Object Explorer to the SQL Server you use for SQL-NS development.
2. | In the Object Explorer tree, find the Notification Services folder.
3. | Right-click the Notification Services folder and select New Notification Services Instance.
4. | In the New Notification Services Instance dialog box, click the Browse button to browse for this chapter's sample ICF. In the File Open dialog box, navigate to C:\SQL-NS\Samples\MinimalMultiApplication and select the InstanceConfiguration.xml file. Make sure you open InstanceConfiguration.xml from the correct directory. If you used Management Studio to create the StockBroker instance in the last chapter, the File Open dialog box may start in the StockBroker sample directory, which also contains an InstanceConfiguration.xml file. Caution If the C:\SQL-NS\Samples\MinimalMultiApplication\InstanceConfiguration.xml file appears to be missing on your system, you probably have not completed all the setup steps given in Chapter 2. If this is the case, return to Chapter 2 and make sure you complete the steps in the "Customizing the Source Files for Your Environment" section (p. 39).
5. | The parameters grid in the New Notification Services Instance dialog box will be populated with the parameters from the ICF. You should not have to change any of these values.
6. | Check the box labeled Enable Instance After It Is Created, below the parameters grid.
7. | Click the OK button to begin compiling the instance and its two associated applications. A progress dialog box will appear, listing the various steps being performed. After all steps are complete, you should see the status reported as Success. When this happens, click Close to dismiss the progress dialog box.
8. | Now you should see a new item in the Notification Services folder in the Object Explorer tree: an instance named MinimalMultiApplication.

Figure 4.3 shows what your Management Studio window should look like after the New Notification Services Instance progress dialog box completes. Notice the new item in the Notification Services folder in the Object Explorer.

Figure 4.3. Creating a new SQL-NS instance using the Management Studio tools.

[View full size image]

Examining the Database

After successfully compiling the instance with either nscontrol create or the New Notification Services Instance command in Management Studio, you can examine the database objects created by the SQL-NS compiler:

1. | Open Management Studio and connect the Object Explorer to your SQL Server, if you have not already done so.
2. | Expand the Databases folder and find the database called MinimalMultiApplication.
3. | Expand the various folders under the MinimalMultiApplication database node to see the various tables, views, and stored procedures created by the SQL-NS compiler.

In the "Instance Database Objects" section (p. 98), we'll explore a few of the instance objects in the database in more detail, but for now, you should spend some time just looking at some of them. Focus on the objects qualified with the NSInstance schema name (the schema name specified in the <Database> element in the ICF). Ignore the application database objects (those qualified with the schema names Application1 and Application2) for now. We'll examine application database objects in detail in Chapter 5, "Designing and Prototyping an Application.")

In addition to creating the instance and application database objects, the SQL-NS compiler also installs metadata about the instance in SQL-NS system tables. In the "Instance Metadata" section (p. 100) later in this chapter, we take a closer look at the information in these tables.

Registering the Instance

Compiling the ICF resulted in SQL-NS creating the instance and application database objects in the MinimalMultiApplication database. Now we need to register the instance to install the required Registry keys, the Windows service that will host the SQL-NS engine, and the performance counters that are used to monitor the activities of the instance.

Registering the Instance with the Command-Line Tools

To register the instance with the SQL-NS command-line tools, we use the nscontrol register command. As we did with the nscontrol create command in the previous section, we'll use a batch file to wrap the nscontrol register command and its arguments. Let's examine this file to understand the nscontrol register command syntax. Open the register.cmd file, located in the Samples\MinimalMultiApplication\Scripts subdirectory off your source code base directory, in a text editor.

Like create.cmd, which we looked at in the previous section, register.cmd begins with some environment variable settings that will be required later. Looking further down in the file, you'll see that the call to nscontrol register is as follows:

[View full width]

nscontrol register -n %INSTANCE_NAME% -s %SQL_SERVER% -service -serviceusername "
  %NS_SERVICE_USER%" -servicepassword "%NS_SERVICE_PASSWORD%" %SQL_USER_INFO%

The -n argument specifies the name of the instance to register. This is obtained from an environment variable, which, as before, is established in one of the earlier setup scripts that register.cmd invokes. The -s argument specifies the name of the SQL Server on which the instance and application databases are located, again obtained from an environment variable. Note that the value passed for this argument should be the same as specified in the <SqlServerSystem> element of the ICF. When we look at the ICF in detail, you'll see that it is the same because the <SqlServerSystem> element's value was obtained from the _SqlServer_ parameter, for which we used the same %SQL_SERVER% environment variable as a value.

The -service argument instructs SQL-NS to install the SQL-NS Windows service to host the SQL-NS engine. This argument exists because there are some cases where you will want to register the instance but not install the service: for example, if you have a dedicated web server to host your subscription management interface or if you are hosting the SQL-NS engine in your own application process, as described in Chapter 17. The -serviceusername and -servicepassword arguments specify the name and password of the Windows account under which the service should run.

In Chapter 2 we created an account for the service and stored the username in an environment variable. We use that environment variable to obtain the username here and pass it to the -serviceusername argument. The code at the beginning of the register.cmd file prompts for the password for this account and stores it in the %NS_SERVICE_PASSWORD% environment variable. This variable is then used to supply a value for the -servicepassword argument. After it has been used in the call to nscontrol register, the %NS_SERVICE_PASSWORD% environment variable is cleared out (for obvious security reasons).

When you're done examining the contents of the register.cmd file, use the following instructions to run it and register the instance with the nscontrol register command:

1.
Open a Notification Services Command Prompt on your development machine and navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MinimalMultiApplication\Scripts

2.
Run register.cmd. The script prompts you for the password of the Windows account you created for running the service. (Note that this is the password of the Windows user, not the SQL user you created if you're using SQL Server Authentication.) If you're using SQL Server Authentication, the script also prompts you for the SQL password of the test account you created in Chapter 2.

Figure 4.4 shows nscontrol register invoked on the command line. The command prints out status messages indicating that the Registry entries, Windows service, and performance counters are installed successfully.

Figure 4.4. Running nscontrol register on the command line.

[View full size image]

Registering the Instance with the Management Studio Tools

To register the instance with the graphical SQL-NS tools in Management Studio, you use the Register Instance command. This command is launched from the context menu that appears when you right-click a particular instance item in the Notification Services folder in the Object Explorer tree. (Note that this is different from the New Notification Services Instance command, which is always launched from the context menu on the Notification Services folder itself.) When you invoke the Register Instance command, Management Studio opens the Register Instance dialog box, in which you can choose to install the SQL-NS Windows service and supply the required credentials.

It's important to note that even if you're connected to a remote SQL server in the Object Explorer, the Register Instance command always creates the instance registration (including the Registry keys, the Windows service, and the performance counters) on the local computer on which you're running Management Studio. To register an instance on another computer, you have to physically run Management Studio or nscontrol register on that computer.

Follow these instructions to register this chapter's sample instance with the Management Studio tools:

1. | Open Management Studio and connect the Object Explorer to your SQL Server.
2. | In the Object Explorer tree, find the Notification Services folder and the MinimalMultiApplication item in it. The MinimalMultiApplication item was created when you compiled the instance with either nscontrol create or the New Notification Services Instance command in Management Studio. If you've compiled the instance but don't see an item for it in the Notification Services folder, try refreshing the folder's contents by right-clicking it and selecting Refresh.
3. | Right-click the MinimalMultiApplication instance and select Tasks and then Register from the context menus. The Register Instance dialog box opens.
4. | Check the box labeled Create Windows Service.
5. | Beneath the Service Logon label, enter the name and password for the Windows user account you created in Chapter 2 to run the SQL-NS service. If your service user account is a domain account, don't forget to qualify the account name with the domain name.
6. | If you're using SQL Server Authentication, select the option button labeled SQL Server Authentication beneath the Authentication label. Enter the SQL login name and password for the test account you created in Chapter 2. If you're using Windows Authentication, leave the Windows Authentication option button selected.
7. | Click OK to begin registering the instance. A progress dialog box listing the various registration operations will appear. When registration completes successfully, click the Close button to dismiss the progress dialog box.

Figure 4.5 shows a Management Studio window immediately after the Register Instance command has completed. The progress dialog box indicates that the Registry entries, Windows service, and performance counters are installed successfully.

Figure 4.5. Registering a SQL-NS instance using the Management Studio tools.

[View full size image]

Granting Permissions

When you registered the instance using the steps in the previous section, you configured the service to run under a specific Windows account. If you're using SQL Server Authentication, you also configured the service to connect to the database with a SQL username and password. You now have to configure database permissions so that the Windows service can access the instance's database at runtime. You also need to assign the appropriate filesystem permissions to the service's Windows account so that components such as the file delivery protocol can read and write the necessary files. This section explains how to correctly configure both database and filesystem permissions for a SQL-NS instance.

Granting Database Permissions

When you completed the instructions in Chapter 2, you created a SQL Server login for the SQL-NS Windows service (see the "Setting Up SQL Permissions and Accounts" section, p. 32). If you're using Windows Authentication, the login you created was for the service's Windows user account. If you're using SQL Server Authentication, you created a SQL Server login with a password.

You now need to set up the service's login as a user in the instance's database. This will allow the service to use the database when it establishes a connection to the SQL Server. You also need to add the service user to specific SQL-NS database roles. SQL-NS defines a set of database roles corresponding to the various operations that can be done with the instance and application database objects. For example, there is a role for event submission and another role for subscription management. The permissions on the objects in the database are set so that only users in the appropriate roles can access them. The complete set of SQL-NS database roles is documented in the SQL-NS Books Online. However, in this case, you just need to place the service user in the NSRunService role, which allows it to perform the operations required of the SQL-NS engine components.

Note

This might seem like a lot of work to get a simple instance running, but it's necessary to keep your system secure. Of course, it would be much easier if you just let your service run under an Administrator account and granted full permission to your databases. But then you run the risk that someone tampering with the service would have unrestricted access to your data. Instead, it's wise to always grant only the minimum set of permissions needed. This takes more time and involves more steps, but it's worth the trouble.

All the samples that accompany this book come with a T-SQL script called SQLPermissions.sql. This script executes the commands needed to set up the required database permissions. For the sample we're working with in this chapter, you'll find SQLPermissions.sql in the C:\SQL-NS\Samples\MinimalMultiApplication directory.

Note

SQLPermissions.sql is one of the files that was generated from a template when you ran the setup script described in the "Customizing the Source Files for Your Environment" section (p. 39) in Chapter 2. The contents of the script will be different, depending on whether you set up your system to use Windows Authentication or SQL Server Authentication.

Open the file C:\SQL-NS\Samples\MinimalMultiApplication\SQLPermissions.sql in Management Studio to examine its contents. The first line in the file changes context to the MinimalMultiApplication database:

USE MinimalMultiApplication

Next, the script invokes the CREATE USER command to create a user in this database for the service login. On my system, I'm using Windows Authentication and my service runs under the PATHERNET\SQL-NS_Service domain account, so the line reads:

CREATE USER [PATHERNET\SQL-NS_Service]

On your system, this line should specify the name of your service Windows account if you're using Windows Authentication, or the name of your SQL test account if you're using SQL Server Authentication.

Finally, the script adds the service user to the NSRunService database role. On my system, this line reads

EXEC sp_addrolemember 'NSRunService', 'PATHERNET\SQL-NS_Service'

Again, the username may be different on your system.

Run the SQLPermissions.sql script to set up the database permissions on your system. You can run it directly in a Management Studio query window or you can use the batch file, grant_permissions.cmd, in the sample's scripts directory (C:\SQL-NS\Samples\MinimalMultiApplication\Scripts) to run it from the command line. The grant_permissions.cmd batch file invokes the sqlcmd command-line utility to run the same SQLPermissions.sql script.

Granting Filesystem Permissions

In this chapter's sample (and the one in the previous chapters), the SQL-NS engine is configured to use the file delivery protocol to write notifications to a file. The Windows account of the SQL-NS service hosting the engine therefore needs write permissions to the notifications file.

When you create a SQL-NS instance, you must determine the filesystem permissions needed by the SQL-NS engine, based on the engine components that your instance and its applications will use. In the samples we've looked at so far, the file delivery protocol is the only component that needs access to the filesystem, so we need to grant the service user write access to the notifications file. If we were using other SQL-NS components, such as the FileSystemWatcher event provider (covered in Chapter 8), we'd have to grant other permissions as well.

You can grant filesystem permissions using Windows Explorer: right-click the relevant file or directory, choose Properties from the context menu, and go to the Security tab in the properties dialog box. From there you can assign the appropriate permissions to the service account.

You can also assign permissions from the command line, using the cacls utility. The setup scripts that were run when you completed the instructions in the "Customizing the Source Files for Your Environment" section (p. 39) in Chapter 2 already invoked cacls to assign the appropriate filesystem permissions for all the book's samples to the service account you specified. If you want to see how this was done, open the setup.cmd batch file in this chapter's scripts directory and look at the calls to cacls near the end.

Enabling the Instance

As mentioned earlier in this chapter, when you first create an instance, all components within it are configured as disabled. This is to prevent any components from running without your explicit consent.

SQL-NS allows you to enable (or disable) each running part of the instance separately. You can enable applications individually and even choose to enable individual components within each application (for example, a specific event provider or distributor).

Enabling the Instance with the Command-Line Tools

To enable components of the instance from the command line, use the nscontrol enable command. As you might expect, the nscontrol disable command is used to disable components.

In the Samples\MinimalMultiApplication\Scripts directory, you'll find a script called enable.cmd that invokes to nscontrol enable. Take a look at the nscontrol enable syntax by opening this file in a text editor.

In enable.cmd, the call to nscontrol enable appears as follows:

nscontrol enable -n %INSTANCE_NAME% %SQL_USER_INFO%

For simplicity, this command just enables the whole instance. The -n argument provides the instance name. If we wanted to enable only certain parts of the instance, we'd have to specify additional arguments to nscontrol enable to identify the components the command should affect. These arguments are explained in the "Enabling and Disabling Components" section (p. 492) in Chapter 14.

Again, the %SQL_USER_INFO% environment variable is used. As in previous commands, if you're using SQL Server Authentication, it supplies values for the -sqlusername and -sqlpassword arguments; otherwise, it evaluates to an empty string.

Run the enable.cmd script:

1.
Open a Notification Services Command Prompt on your development machine and navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MinimalMultiApplication\Scripts

2.
Run enable.cmd. If you're using SQL Server Authentication, the script will prompt you for the SQL password for the development account you created in Chapter 2. If you're using Windows Authentication, you will not be prompted.

When it completes, the enable command prints out the status of the various components in the instance. At this point, all should say either Enabled or Enable Pending. Those that say Enable Pending will become enabled when the SQL-NS engine starts.

Enabling the Instance with the Management Studio Tools

When you first create an instance in Management Studio, you have the option of enabling it at the same time. To do this, you check the Enable Instance After It Is Created box, below the parameters grid in the New Notification Services Instance dialog box. You were instructed to do this in step 6 of the instructions in the "Compiling the ICF with the Management Studio Tools" section (p. 87), earlier in this chapter. If you did not do this, or if you disable the instance at some later time and want to reenable it, you can use the facilities described in this section.

Note

Though you can choose to enable an instance when you create it in Management Studio, there is no equivalent option in the command-line nscontrol create command.

To enable a whole instance from Management Studio, launch the Enable Instance command from the context menu on the instance item in the Notification Services folder. When you invoke this command, you'll be prompted for confirmation. If you answer Yes in the confirmation dialog box, the command will proceed and no other visual feedback will be given.

Unfortunately, there is no immediate visual indication in the Object Explorer as to whether a particular instance is enabled. To check the enabled status of an instance and its components, you have to open the properties dialog box for the instance from its context menu (right-click the instance in the Object Explorer and choose Properties).

The properties dialog box has several pages; you can select a page to view from the list in the left pane of the dialog box. The Applications page shows you the status of the various applications in the instance and their components. You can change the status of individual components via the Enabled check box on each line in the components grid. The Subscribers page in the properties dialog box lets you view and change the enabled status for subscriber management. To enable the instance we created in this chapter, use the following instructions. These steps will enable the whole instance. There's no harm in enabling an instance that's already enabled, so feel free to perform these steps, even if you enabled the instance at the time you created it.

1. | Open Management Studio and connect the Object Explorer to your SQL Server.
2. | In the Object Explorer tree, find the Notification Services folder and the MinimalMultiApplication item in it.
3. | Right-click the MinimalMultiApplication instance and select Enable from the context menu.
4. | Answer Yes when the confirmation dialog box appears.

Starting the Instance

Now that we've created the instance, registered it, granted database permissions, and enabled the required components, we are ready to start the instance. This amounts to starting the Windows service that was installed for the instance when we registered it.

The Windows service for a given instance is always named NS$<InstanceName> where <InstanceName> is the name of the instance. Because in this chapter, our instance is called MinimalMultiApplication, the service is called NS$MinimalMultiApplication.

To start the instance's service, you can use any of the standard mechanisms for managing Windows services. These include the net start command and the Services applet in Administrative Tools. For example, to start the service from the command line using net start, issue the following command:

net start NS$MinimalMultiApplication

You should see messages indicating that the service is starting and finally one that says it started successfully.

Tip

If your service did not start successfully, look in the Application Event Log for error messages. (Open Administrative Tools from your Start menu and choose Event Viewer; open the Application log and look for messages that have NotificationServices listed as the source.) In general, the Application Event Log is the first place you should look if you have problems with your SQL-NS applications.

SQL Server 2005's toolset introduces two new ways you can start and stop instances: using Management Studio and using the SQL Server Configuration Manager. From Management Studio, right-click an instance in the Object Explorer and select Start from the context menu. This has the advantage of starting the instance's service on all computers on which it is deployed (in the case of a multimachine deployment, as described in the "Deployment of SQL-NS Instances on Multiple Servers" section, p. 102, later in this chapter). The SQL Server Configuration Manager (launched from the Start menu, All Programs, Microsoft SQL Server 2005, Configuration Tools, SQL Server Configuration Manager) provides a central interface for managing all the services associated with SQL Server 2005 (including SQL-NS instance services). In SQL Server Configuration Manager, you can select the instance service from the list and use either the buttons in the toolbar or the context menus to start and stop it.

Choose one of the methods described in this section and start the service for this chapter's instance on your system. Whichever method you used, if your service started successfully, you now have a running instance. In its current form, it's not very exciting because there are no events or subscriptions and, therefore, no notifications. We'll get to adding those things in Chapter 5. For now, you've learned the basic steps to get the instance running.

If this seemed like a lot of work for no results, be assured that you're over the hardest part. You've built the basic framework in which any application can be implemented. From now on, all the code you'll see in the next few chapters adds functionality that you can observe and test. Whenever you build SQL-NS instances and applications going forward, you'll need to use the same patterns and techniques you learned in this chapter. However, you'll probably automate most of the steps and they'll go by much faster.


Examining the Parts of the Instance

In this section, we'll take a more in-depth look at the elements of the instance we created. If you're eager to get coding, you can skip this for now, but it's worth reading to solidify your understanding.

Instance Database Objects

The instance database objects are used to store subscriber data, information about the applications in the instance, the installed delivery protocols, the configured delivery channels, as well as common reference information needed by all applications in the instance. The following sections describe some of the key instance database objects.

Subscriber Data

SQL-NS applications are about providing notification services to subscribers. As you saw in Chapter 3, the subscriptions are stored as rows in tables. As you might expect, the data about the subscribers themselves is also stored in tables.

However, instead of subscriber data being stored in each application, it is stored at the instance level. This means that all applications within a single instance share a single set of subscribers. Each application maintains its own subscriptions, but the subscribers are common. The shared subscriber data includes the subscriber ID, enabled status, timestamps to track when the subscriber record was created and modified, and delivery device information.

Sharing subscriber data makes sense because applications grouped in a single instance are usually related, and the subscribers are likely to have subscriptions in more than one of those applications. Sharing the subscribers avoids duplication of data.

Two main tables in the instance schema relate to subscribers: NSSubscribers and NSSubscriberDevices. As you might expect from their names, the first table stores information about subscribers themselves, and the second stores information about the devices on which subscribers receive notifications. In Chapter 7, we'll examine subscribers and subscriber devices more closely, but for now, you should explore the structure of these tables in Management Studio. (The instructions in the "Examining the Database" section, p. 88, earlier in this chapter, explain how to view the instance database objects in the Object Explorer.)

Application Configuration

The instance's database contains the list of applications in the instance, as well as the enabled state of each. This allows the Windows service to properly manage the instance's applications. The application information is stored in the NSApplicationNames table in the instance schema. Execute the following query in a Management Studio query window to see the list of applications:

USE MinimalMultiApplication
SELECT * FROM [NSInstance].[NSApplicationNames]

This should return two rows: one for each of the two skeleton applications in our sample instance.

Delivery Protocols and Delivery Channels

The instance's database also contains the list of installed delivery protocols that applications in the instance can use to send notifications. Delivery protocols are components that implement the required communication mechanisms to enable notification delivery to various network endpoints. For example, an SMTP delivery protocol component implements the message exchange and format defined by SMTP for sending email. Delivery protocol implementations are components that can plug into the SQL-NS framework. SQL-NS ships with some protocol implementations built in, but others can be developed and installed later. Because the list of installed protocols is kept in the instance, the protocols are available to all the applications in the instance.

In addition to the delivery protocols, the instance database also maintains the list of configured endpoints to which notifications can be sent. For example, it may list the SMTP servers in your network that are used for sending email notifications. The configuration information usually includes the name of the endpoint and authentication information used to negotiate a session with it. Each configured endpoint is called a delivery channel. Both delivery protocols and delivery channels are discussed in more detail in Chapter 10; for now, just note that the NSProtocols, NSDeliveryChannels, and NSDeliveryChannelArguments tables in the instance schema store configuration information about them.

Common Reference Information

The instance schema also contains some tables used by all applications for reference information. For example, tables store time zone information (NSTimeZones, NSTimeZoneNames, and NSTimeZoneDstOffsets) and another lists the supported locales (NSSubscriberLocales).

Application Database Objects

As we saw when we created the instance, the SQL-NS compiler creates a database schema for each application in the instance. This schema contains tables used to store the configuration information about the application, as well as the application's events, subscriptions, and notifications. We'll examine the contents of an application schema in detail in Chapter 5.

Instance Metadata

When the SQL-NS compiler creates the instance and application database objects, it also installs metadata about the instance in the SQL-NS system tables. This metadata is used by the SQL-NS tools to enumerate the instances on a server (the list of instances you see in the Object Explorer is obtained from the metadata tables). The SQL-NS metadata tables are in the msdb system database, in the NS90 schema. You should never add, remove, or alter the data in these tablesthey are intended for system use only.

You can view the instance metadata by executing the following query in a Management Studio query window:

USE msdb
SELECT * FROM [NS90].[NSInstances]

This query will result in a row of data for each instance on your SQL server.

Because it's critical that the instance metadata maintains an accurate picture of the instances on a server, you should never attempt to remove an instance by dropping its database. If you do this, the metadata for the instance will remain, even though its database objects are gone. Instead, you should always delete instances by using the nscontrol delete command, or via Management Studio by right-clicking an instance in the Object Explorer and selecting Tasks, followed by Delete. Both tools will clean up the metadata appropriately, after the instance has been removed. See the "Cleaning Up the Instance" section (p. 112) near the end of this chapter for more information on removing a SQL-NS instance.

Tip

If you do accidentally remove a SQL-NS instance database, you can remove the associated metadata by running nscontrol delete. The command will display an error message indicating that the instance database could not be found, but it will also remove the instance's metadata if it is present. See the documentation for nscontrol delete in the SQL-NS Books Online for more details.

If you need to move a SQL-NS instance's database from one server to another, you can rebuild the metadata for the instance on the new server using the nscontrol repair command. For more information on nscontrol repair, see the SQL-NS Books Online.

Registry Keys

SQL-NS stores almost everything about an instance and its applications in the database. But, for the running components of the instance to get to that information, they have to be able to connect to the database: They need to know the database server name, and the SQL username and password to supply if SQL Server Authentication is used.

Obviously, this information can't be stored in the database (we'd have a chicken-and-egg problem if it was), so it's stored instead in a well-known place in the Registry. Given an instance name, SQL-NS components can look up the database connection information for the instance. Registering the instance using either nscontrol register or the Register Instance command in Management Studio places this information in the Registry. Note that if a SQL username and password are stored, they are encrypted.

The instance Registry keys are stored under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\Services\NotificationServices\Instances. When you register an instance, SQL-NS creates a key in this Registry location, with the instance name. The database connection information is then stored as a set of values beneath this key. Figure 4.6 shows these Registry keys for the instance created in this chapter (viewed using the regedit tool).

Figure 4.6. The instance Registry keys shown in the Registry Editor.

[View full size image]

Caution

You should never manually change the instance registration in the Registry. Instead, always use the nscontrol or Management Studio commands to manipulate the instance registration.

Multiple Side-By-Side Instances

SQL-NS allows several instances to exist side-by-side on a single computer. Logically, you can treat each instance as a separate installation of SQL-NS, although all instances on a single computer usually run off the same installed SQL-NS program files.

Each instance is independent of all other instances; instances can be administered separately and configured under different security accounts. Different instances on the same computer can even run different versions of SQL-NS. Whenever you do a SQL-NS operation, you always do it against some particular named instance.


Deployment of SQL-NS Instances on Multiple Servers

The deployment of a SQL-NS instance can span multiple servers. The instance database and the SQL-NS engine do not need to be located on the same server (although they certainly can be, if that suits your deployment). Figure 4.7 shows various deployment configurations, including some on which the database and the SQL-NS engine are on different servers.

Figure 4.7. Various SQL-NS instance deployments across servers.

[View full size image]

The split of SQL Server and the SQL-NS engine onto two separate machines is common. Many organizations share their SQL Server among many applications and like to keep the database server dedicated only to hosting databases. Also, this is a convenient split of resources for the way SQL-NS tends to operate. In many SQL-NS applications, about half the work tends to be done in the database (matching events and subscriptions), and the other half done in the SQL-NS Service (formatting and delivering notifications).

There are still more possibilities for deploying the instance on multiple servers if you are using the Enterprise Edition of SQL-NS. With Enterprise Edition, the various running components of each application can be configured to run on different servers. In other words, you can configure the event providers, the generator, and the distributors of a given application all to run on separate servers. Or you can have all the components for one application run on one server, and all the components for another application on a separate server. They are all still part of the same instance, but the deployment is spread out. Each server running application components runs its own copy of the SQL-NS service, and that service just hosts the components configured to run on that machine. Chapter 13 describes the various deployment options in detail. If you are running the Standard Edition of SQL-NS, all running components of the instance must be on the same machine, although the SQL Server can be on a separate machine.

SQL-NS Instances and SQL Server Instances

If you've worked with instances in SQL Server, the basic instance concept should be familiar. SQL Server lets you also create independent, named configurations and calls these instances. SQL Server instances do not necessarily map 1:1 to SQL-NS instances.

A single SQL Server instance may host databases from several SQL-NS instances. However, a single SQL-NS instance stores all its data on a single SQL Server instance.


The Instance Configuration File in Detail

As described earlier, to create an instance, you write an Instance Configuration File (ICF). I showed the ICF for the instance created in this chapter earlier, but we did not look at its contents.

Table 4.1 shows some of the key elements in the ICF. In this section, we'll take a closer look at each of these.

Table 4.1. XML Elements in the ICF

XML Element

Purpose

<NotificationServicesInstance>

Root element that defines the instance.

<InstanceName>

Name for the instance.

<ParameterDefaults>

Provides default values for parameters used in the ICF.

<SqlServerSystem>

Identifies the SQL Server on which the instance and application database objects will be created.

<Database>

Specifies the name of the database and schema in which the instance's database objects will be deployed when the ICF is compiled.

<Applications>

Lists the applications in the instance.

<Protocols>

Declares custom delivery protocols used in the instance.

<DeliveryChannels>

Defines the configured endpoints to which applications in the instance can send notifications.

Note

Many of the ICF elements are optional. Not every ICF you encounter will have all the elements shown in Table 4.1

The ICF in this chapter is basic and can be a starting point for any instance you create. My intent here is not to present an exhaustive reference of every XML element and option that the ICF supportsthe SQL-NS Books Online already do a great job of that. Instead, I'll introduce you to the key elements that you need to get started and mention some others that might be useful as you develop and deploy your own instances. Every element described here is also documented in detail in the SQL-NS Books Online. Here I simply describe the intended purpose of each element and how to use it; I do not cover the syntactical details such as naming restrictions and formats, which are covered in the SQL-NS Books Online.

In the following sections, we'll look at one piece of the ICF at a time. Each of these pieces is intended to highlight just a single part of the ICF content but will likely not compile by itself. I indicate missing code (code shown in other sections) by means of the ellipsis (...).

Parameters

Before actually delving into the code, I want to first expand on the notion of parameters in Notification Services source files. (You saw parameters used when examining the nscontrol create command line earlier.) Proper use of parameters allows you to make your instance and application definitions flexible, and makes administration tasks much easier.

Parameters are simply tokens in source files that the SQL-NS compiler substitutes with values you supply at compile time. Consider a case where parameters are useful: in the ADF, you have to specify the name of the server on which the application components (such as the generator) should run. If you put the server name directly in the ADF, you'd have to edit the ADF if you wanted to change it. Worse, if this server name was used in many places in the ICF and ADFs, you'd have to remember to find and change all of them.

Instead of putting the server name in the ADF directly, you can use a parameter: a token, such as %_NSServer_%. At the time of invoking the SQL-NS compiler, you'd have to specify a value for this parameter. The SQL-NS compiler then substitutes this value for every occurrence of %_NSServer_%. In SQL-NS ICFs and ADFs, any token between percent (%) signs is treated as a parameter.

By using a parameter for the server name, if you wanted to deploy the application components on a different server, you would have to pass a different value for the parameter when you invoked the compiler. You would not need to edit your ICF and ADFs; you could reuse them unmodified for several servers. Also, using a parameter ensures that all references to the server name would be changed at once.

Parameters used in the ICF are passed directly to the compiler, either as arguments to nscontrol create on the command line or as values typed into the parameters grid in the New Notification Services Instance dialog box in Management Studio. But these parameters are not automatically usable in the ADFs referenced by the ICF. Any parameters used in an ADF must be declared and assigned values when that ADF is referenced in the ICF. You'll see this done in the section "The Applications" (p. 109), which describes the <Application> element of the ICF. You can assign the value of an ICF parameter to an ADF parameter to "pass through" the ICF parameter to the ADF, as shown in Figure 4.8.

Figure 4.8. Parameters, the ICF, and the ADFs.

[View full size image]

As shown in Table 4.1, the ICF schema defines an optional element called <ParameterDefaults> that allows you to specify default values for parameters. Parameters assume the default values as declared in the <ParameterDefaults> element, unless you specify different values when you invoke the SQL-NS compiler (regardless of whether you use the command-line or Management Studio tools). In the case of Management Studio, when you load an ICF in the New Notification Services Instance dialog box, the parameters grid is initialized with the default values from the ICF's <ParameterDefaults> element.

The <ParameterDefaults> element from this chapter's ICF is shown in Listing 4.2.

Listing 4.2. The <ParameterDefaults> element of the ICF

<NotificationServicesInstance>
...
  <ParameterDefaults>
    <Parameter>
      <Name>_SQLServer_</Name>
      <Value>... Your SQL Server Name ...</Value>
    </Parameter>
    <Parameter>
      <Name>_NSServer_</Name>
      <Value>%COMPUTERNAME%</Value>
    </Parameter>
    <Parameter>
      <Name>_InstanceBaseDirectoryPath_</Name>
      <Value>C:\SQL-NS\Samples\MinimalMultiApplication</Value>
    </Parameter>
  </ParameterDefaults>
...
</NotificationServicesInstance>

Notice that on your system, default values for the SQL Server name and the instance base directory path will be customized for your environment. The parameter default values were obtained from the environment variable settings you made when you set up your development environment in Chapter 2 and were written into the ICF when it was generated from a template during the setup stage. Having the correct parameter default values for your environment in the ICF allowed you to create the instance in Management Studio without needing to alter any of the values in the parameters grid.

The value for the _NSServer_ parameter is obtained directly from the environment variable %COMPUTERNAME%. SQL-NS allows you to use environment variable values like this in the ICF. Because %COMPUTERNAME% is a standard environment variable that will be set on every system, we can safely use it. We could not use the other environment variables you created in Chapter 2 directly because they are set only when going through the command-line environment, not when you use Management Studio.

Note

If a parameter is used in the ICF, no value is supplied for it when the compiler is invoked, and no default is given; the ICF will not compile and you'll see an error message from the compiler.

The Root Element

The ICF is an XML document referencing the ICF schema with <NotificationServicesInstance> as the root element, as shown in Listing 4.3.

Listing 4.3. The Outline of the ICF

[View full width]
<?xml version="1.0" encoding="utf-8"?>
<NotificationServicesInstance xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http
  ://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com
  /MicrosoftNotificationServices/ConfigurationFileSchema">
...
</NotificationServicesInstance>

The first line is the standard XML processing directive. Next is the root element, with all the namespace specifiers. Last is the closing tag for the root element. All the code in the rest of this chapter goes in between the open and close tags of this root element.

Note

The next sections of this chapter show this enclosing root element of the ICF, but without all the schema namespace specifiers. This is simply for brevity and readability; in a real ICF, the namespace specifiers should always be present.

The Instance Name and SQL Server

Inside the ICF's root element, the instance name and target SQL Server are specified by means of the <InstanceName> and <SqlServerSystem> elements, respectively. These are shown in Listing 4.4.

Listing 4.4. Specifying the Instance Name and SQL Server in the ICF

<NotificationServicesInstance>

  <InstanceName>MinimalMultiApplication</InstanceName>
  ...
  <SqlServerSystem>%_SQLServer_%</SqlServerSystem>
  ...
</NotificationServicesInstance>

The SQL Server name is given as a parameter, %_SQLServer_%. When we compiled the ICF, the parameter was substituted with the correct value for the SQL Server name (if you used nscontrol create, the parameter value was passed in on the command line; if you used Management Studio, the parameter default value was used). On my system, the SQL Server is called PATHERNETMCE. With parameter substitution, the preceding code would be equivalent to

<NotificationServicesInstance>
  ...
  <InstanceName>MinimalMultiApplication</InstanceName>
  ...
  <SqlServerSystem>PATHERNETMCE</SqlServerSystem>
  ...
</NotificationServicesInstance>

The value in the <SqlServerSystem> element can also contain a SQL Server instance name if necessary. In my case, I specified a server name because I'm connecting to the default (unnamed) SQL Server instance. If you're using a named SQL Server instance, you could specify a SQL Server name of the form ServerName\InstanceName in the <SqlServerSystem> element.

The Database Element in the ICF

Like the ADF, the ICF can specify a <Database> element. The database and schema name supplied in this element control where the instance database objects are installed. Listing 4.5 shows the <Database> element from this chapter's ICF.

Listing 4.5. Specifying the Database and Schema Name in the ICF

<NotificationServicesInstance>
  ...

  <Database>
    <DatabaseName>MinimalMultiApplication</DatabaseName>
    <SchemaName>NSInstance</SchemaName>
  </Database>
  ...
</NotificationServicesInstance>

In this chapter's sample, and the one in the previous chapter, the ICF and ADFs specified the same database name in their <Database> elements. This resulted in the instance and application database objects being deployed in the same database, isolated in different schemas.

In many cases, keeping all the objects in a single database is desirable, but you can also put them in separate databases if you prefer. To do this, specify different database names in the <Database> elements in the ICF and ADFs. If you do use more than one database, you must grant the SQL-NS engine permission to access each of them. To do this, repeat the steps outlined earlier in the "Granting Database Permissions" section (p. 93) for each database.

The <Database> element is optional in both the ICF and the ADF. If you omit the <Database> element from the ICF, the instance database objects will be created in the dbo schema in a dedicated database. The name of this database will be the instance name, followed by NSMain. For example, if we'd omitted the <Database> element in this chapter's ICF, the instance database objects would have been placed in a database called MinimalMultiApplicationNSMain. If you omit the <Database> element in an ADF, then the application's database objects will be placed in the dbo schema in another dedicated database. The name of this database will be the instance name, concatenated with the application name. For example, the dedicated database for the Application1 application in this chapter's instance would be called MinimalMultiApplicationApplication1 had we omitted the <Database> element from the ADF.

Note

Though I'm not showing them here, other subelements within the <Database> element allow you to customize how the databases are created. For example, you can specify filegroups for the data and log files, as well as the collation. See Chapter 12, "Performance Tuning," and the SQL-NS Books Online documentation for the <Database> element for more details.

The Applications

The instance is a container for a set of related applications, and the ICF lists those applications. The list appears in the <Applications> element, as shown in Listing 4.6.

Listing 4.6. Specifying the Applications in the Instance

<NotificationServicesInstance>
  ...
  <Applications>
    <Application>
      <ApplicationName>Application1</ApplicationName>
      <BaseDirectoryPath>
        %_InstanceBaseDirectoryPath_%\Application1
      </BaseDirectoryPath>
      <ApplicationDefinitionFilePath>
        ApplicationDefinition.xml
      </ApplicationDefinitionFilePath>
      <Parameters>
        <Parameter>
          <Name>_NSServer_</Name>
          <Value>%_NSServer_%</Value>
        </Parameter>
        <Parameter>
          <Name>_ApplicationBaseDirectoryPath_</Name>
          <Value>%_InstanceBaseDirectoryPath_%\Application1</Value>
        </Parameter>
      </Parameters>
    </Application>
    <Application>
      <ApplicationName>Application2</ApplicationName>
      <BaseDirectoryPath>
        %_InstanceBaseDirectoryPath_%\Application2
      </BaseDirectoryPath>
      <ApplicationDefinitionFilePath>
        ApplicationDefinition.xml
      </ApplicationDefinitionFilePath>
      <Parameters>
        <Parameter>
          <Name>_NSServer_</Name>
          <Value>%_NSServer_%</Value>
        </Parameter>
        <Parameter>
          <Name>_ApplicationBaseDirectoryPath_</Name>
          <Value>%_InstanceBaseDirectoryPath_%\Application2</Value>
        </Parameter>
      </Parameters>
    </Application>
  </Applications>

    ...
  </Applications>
  ...
</NotificationServicesInstance>

The <Applications> element contains an <Application> element for each application in the instance. The fragment in Listing 4.6 shows the <Application> element for both applications in this chapter's instance.

The first subelement of <Application> is <ApplicationName>, which, as you might guess, specifies the name of the application. In this simple example, the application is called Application1, although you'd most likely have a more descriptive name for a real application.

The <BaseDirectoryPath> element specifies the root of the directory tree in which all the application's files are located. These files include the ADF and any supporting files required by the application. In this example, the value of the <BaseDirectoryPath> element is derived from a parameter value, %_InstanceBaseDirectoryPath_%.

This is because of the way I've chosen to lay out the files of the instance. I have one root directory for the entire instance, and every file belonging to the instance either exists in this directory or in a subdirectory off it. The parameter, _InstanceBaseDirectoryPath_, provides the name of the root directory of the instance. Each application is in a directory off this, named with the application name. So, Application1 is in %_InstanceBaseDirectoryPath_%\Application1.

Note that nothing in SQL-NS mandates this directory layout. This is just one that makes sense to me (and is common in many SQL-NS deployments I've seen).

The <ApplicationDefinitionFilePath> element specifies the path to and the name of the application's ADF, relative to the application's base directory path in the previous element. Because our ADF is located in the root of the application's directory, this is just a filename. When compiling the instance, SQL-NS goes through each of the application declarations and compiles the ADF, located by appending the <ApplicationDefinitionFilePath> to the <BaseDirectoryPath>.

The next element is the <Parameters> element. This is where the input parameters to the ADF are specified. Each contains a <Name> and <Value> subelement that contains, as you might expect, the name and the value of the parameter, respectively.

Because each application has its own <Parameters> element, you can specify different values for the same parameter name in each application. In the example shown here, the first parameter, _NSServer_, is simply a pass-through of the value of the instance parameter with the same name. So, %_NSServer_% is specified as the value expression. The second parameter, _ApplicationBaseDirectoryPath_, is derived from the instance base directory path. This is essentially the same value as that given for the <BaseDirectoryPath> element, so you might wonder why it's being created twice.

When specified as a value for <BaseDirectoryPath>, it's simply a value for an ICF element. Here it's being specified as a value to a parameter that will be passed into the ADF. This is needed because the elements in the ADF use this to specify the location of various files. Unless passed in as an explicit parameter, the ADF has no way to refer to the value given for <BaseDirectoryPath>. It's unfortunate that we have to derive this value twice, because if we changed it, we'd have to change it in two places. It would be nice if SQL-NS allowed us to create new parameters (a bit like local variables) in the ICF. However, this facility doesn't exist, so we have to live with the limitation.

Delivery Channels

The last ADF element we'll look at in this chapter is the <DeliveryChannels> element. As described earlier, a delivery channel is simply a representation of a named, configured endpoint to which notifications can be sent. Every delivery channel uses a delivery protocol to communicate with the endpoint, and this can be one of the built-in protocols, or a custom delivery protocol, developed by you or a third party. Listing 4.7 shows a sample definition of a delivery channel in the <DeliveryChannels> element.

Listing 4.7. Specifying the Delivery Channels

<NotificationServicesInstance>
  ...
  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      <ProtocolName>File</ProtocolName>
      <Arguments>
        <Argument>
          <Name>FileName</Name>
          <Value>
            %_InstanceBaseDirectoryPath_%\FileNotifications.txt
          </Value>
        </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>

    ...
  </DeliveryChannels>
</NotificationServicesInstance>

Each delivery channel is defined in a <DeliveryChannel> element. Subelements specify a name for the delivery channel (<DeliveryChannelName>) and the name of the protocol the delivery channel uses to deliver messages (<ProtocolName>). In this case, the delivery channel uses the File protocol, which is one of the built-in protocols shipped with SQL-NS. If you were using a custom developed delivery protocol, you would have to declare it, give it a name, and then reference that name here. Chapter 10 shows examples of using custom delivery protocols.

Finally, the <DeliveryChannel> element contains a list of named arguments in the <Arguments> element. The values of these arguments are passed to the delivery protocol implementation when the delivery channel is initialized.


Cleaning Up the Instance

The instance and applications we created in this chapter really don't do anything useful, but they were helpful in allowing us to examine the various parts of the system. You really won't need to work with this instance and its applications after this chapter, so you can use the following steps to remove them from your system. This cleans up the databases, Registry keys, and settings associated with the instances.

Much like the instance creation steps, two of these operations are implemented as batch files that invoke nscontrol commands. These commands delete and unregister the instance. Deleting removes the database, and unregistering removes the Registry keys and performance counters and uninstalls the Windows service associated with the instance.

I won't explain them in detail here, but it's worth taking a moment to look inside the batch files referenced in the following steps to see what they are doing. (They're fairly self-explanatory.)

Note

The equivalent operations are also available in Management Studio, via the context menus on the instances in the Notification Services folder in the Object Explorer. Because these are used in much the same way as the other Management Studio tools already covered, I have not provided a detailed explanation of them here.

1.
Open a Notification Services Command Prompt on your development machine and navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MinimalMultiApplication\Scripts

2.
Stop the service by typing the following command:

net stop NS$MinimalMultiApplication

3.
Close any Management Studio query windows you still have connected to the instance's database.

4.
Run delete.cmd. If you're using SQL Server Authentication, the script will prompt you for the SQL username and password for the development account you created in Chapter 2. If you're using Windows Authentication, you will not be prompted.

5.
Run unregister.cmd.


Summary

The instance is the fundamental unit of administration and deployment in SQL-NS. An instance serves as a container for a group of related applications, and every application belongs to some instance. Each SQL-NS instance is an isolated entity and can be configured and administered separately from all other instances.

To create an instance, you write an ICF: an XML document that describes the instance and the applications within it. Using tools provided by SQL-NS, you can create and register a new instance. Creating the instance involves reading the ICF and building the database structures required to run and manage the instance. Registering the instance configures the host computer to run the SQL-NS execution engine for the instance.

It's important to go through the steps in this chapter once to understand what they actually do. However, in practice, you'll usually automate these steps in scripts and probably won't think about them much.


Part II: Notification Services Application Development

IN THIS PART

Chapter 5 Designing and Prototyping an Application

Chapter 6 Completing the Application Prototype: Scheduled Subscriptions and Application State

Chapter 7 The SQL-NS Subscription Management API

Chapter 8 Event Providers

Chapter 9 Content Formatters

Chapter 10 Delivery Protocols

Chapter 11 Debugging Notification Generation


Chapter 5. Designing and Prototyping an Application

IN THIS CHAPTER

  • What Is a SQL-NS Application?

  • Stages in the Application Development Process

  • Supporting the Sample Application

  • Initial Design: Seven Important Questions About Your Application

  • Building the Application Prototype

In Chapter 4, "Working with SQL-NS Instances," we looked at SQL-NS instances, the containers for groups of SQL-NS applications. In this and the next five chapters, we'll take a detailed look at the applications themselves and the application development process.

This chapter covers the first stages of application development: the initial design and the building of a basic application prototype. In Chapters 6 through 10, you'll learn how to transform a prototype into a real application.

To illustrate the process of designing and prototyping with SQL-NS, we'll create a sample application: a notification service for an online music store. We'll begin the design and prototyping of the application in this chapter and continue to refine it in subsequent chapters. In its final form, this application will use the many powerful features of the SQL-NS platform. In addition to showing you the process of designing and prototyping an application, this and the upcoming chapters explain these SQL-NS features so that you can use them in applications of your own.


What Is a SQL-NS Application?

A SQL-NS application is a piece of software that gathers events and subscriptions and matches them to produce notifications. The kinds of events, subscriptions, and notifications, as well as the logic used to determine when a match occurs, are specific to each application. When you build an application, you specify these things for your application's domain. In this chapter, we'll do this for the music store application.

Whereas a SQL-NS instance is a unit of administration and deployment, a SQL-NS application is a unit of functionality. An application encapsulates event, subscription, and notification data, and the logic that relates them.

A complete SQL-NS application has the following functional parts:

  • Event providers that submit events into the application

  • A subscription management interface that adds, deletes, and modifies users' subscriptions

  • A matching component that matches events with subscriptions and generates notifications

  • Content formatters that transform raw notification data into a format presentable to the final recipient

  • Delivery protocols that deliver formatted notifications

These functional parts (illustrated in Figure 5.1) are each responsible for handling one of the phases of processing in SQL-NS applications, described in the section "Component Configuration and the Phases of Processing" (p. 61) in Chapter 3, "The Simplest Notification Application: Stock Quotes."

Figure 5.1. Parts of a SQL-NS application.

[View full size image]

Think of the matching component as the core of the SQL-NS application, to which all the other components connect as inputs or outputs. The matching component is where the logic of the application is implemented. The input and output components communicate with the matching component through an exchange of data.


Stages in the Application Development Process

A fully functional notification application is likely to be a sophisticated piece of software, consisting of several modules with complex interactions and dependencies. At the start, it can be difficult to correctly predict the implementation details of an application in its entirety. For this reason, SQL-NS applications are usually developed in stages. Figure 5.2 shows the various stages in the process.

Figure 5.2. The development process for a SQL-NS application.

[View full size image]

Development begins with a high-level initial design. This design specifies the important characteristics of the application and scopes the problem that the application will solve. The section "Initial Design: Seven Important Questions About Your Application" (p. 124) describes the questions about your application that need to be asked and answered at the design stage.

After the initial design is complete, you then move on to building a prototype. The prototype essentially consists of the application's matching component, shown at the center of Figure 5.1. However, instead of connecting the matching component to the real input and output components that the application will ultimately work with, you use the test facilities that SQL-NS provides to feed sample data into the matching component and observe the output it generates. Using these facilities, you can determine whether the application's matching logic is working correctly, without concerning yourself with the details of the real input and output components.

As you build the prototype, you'll test it, debug problems, and make refinements as necessary. You may go through several iterations before the prototype is complete. In this chapter, we'll begin prototyping the music store application. In the "Building the Application Prototype" section (p. 126), we start developing the ADF and test it with sample data. By the end of this chapter, we'll have a working prototype that implements the basic features of the music store application's matching component. In Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State," we'll complete the prototype by implementing the more advanced features and then test the complete prototype from end to end.

Note

Chapters 7, 8, 9, and 10 cover the next four stages of the application development process: building a subscription management interface, event providers, content formatters, and delivery protocols. Chapters 12, 13, and 14 describe the final stages: performance tuning, deployment, and administration.


Supporting the Sample Application

As mentioned earlier, the application we start to build in this chapter is a notification service for an online digital music store. Our application enables the store's customers to subscribe for notifications about new music added to the store's catalog.

Many online digital music stores are in business todayMSN Music (http://music.msn.com) is one example. The kind of application we're going to build over the next few chapters could be targeted for any of the existing music stores. But because we don't have access to the internals of any of these real online music stores, our application will be built against a fictional music store that we create. We'll have complete control over our fictional music store's data and processing, which we'll need during the testing of our notification application.

This fictional music store we create in this chapter is a simple model of a real music store; it captures only the key elements necessary to build our notification application. Specifically, the model consists of a database that represents the store's catalog of songs and a mechanism to add songs to this catalog. Other parts of a typical music store, such as the business logic and the infrastructure for conducting real e-commerce transactions, are absent from our simple model because they are not relevant to the notification application.

This section describes the design of the model music store database and the steps you'll need to follow to create it on your system. Technically, this isn't part of the normal notification application development process (if you were a developer building a notification application for a real online music store, this infrastructure would already exist). You just need to perform these steps here to support the sample application. After you complete these steps, you'll be ready to start designing and prototyping the notification application in the next section.

The Music Store Database Schema

The music store database used in this chapter is simple. I've designed a basic schema that models the entities and relationships that compose the music store's catalog, as shown in Figure 5.3. Of course, this schema is not nearly sophisticated enough to be used in a real music store, but for the purposes of illustration in this chapter, it is sufficient.

Figure 5.3. Schema for the music store catalog.

As Figure 5.3 shows, the fundamental entity is the song, which has a title field, as well as the date it was added. Songs are related to albums, which also have title and date added fields. Albums are related to artists and genres, which are stored in separate tables.

This simple, normalized schema works well for our purposes, but it has some obvious limitations. For example, it is not possible to model an album that contains songs by different artists. Nor is it possible to model a song that has more than one artist. However, because the goal of this chapter is not to build the best database design for a music store, but rather to show how a notification application can be built against one, we'll ignore these limitations and proceed with this simple schema.

Creating the Music Store Database

I've provided a set of batch files and SQL scripts that you can use to create the music store database on your system and populate it with data. Follow the instructions in this section to set up the music store database on your system. You need to complete these instructions before building the notification application in the next section.

Note

All the instructions in this chapter assume that you have set up your development environment as described in Chapter 2, "Getting Set Up." If you have not already done this, go back to Chapter 2 and follow the instructions for setting up your system before continuing here.

As in previous chapters, I assume that you installed this book's source code in the C:\SQL-NS directory. If you installed the source code in a different directory on your system, replace C:\SQL-NS with the name of that directory in all the following instructions.

If you're using SQL Server Authentication, many of the scripts in this chapter's instructions prompt you for the SQL password of the development or test account set up in Chapter 2.

1.
From a Notification Services Command Prompt on your development machine, navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run create_music_store_database.cmd. If you receive an error message when running this batch file, refer to the caution box in this section for possible remedies.

This batch file calls sqlcmd to run two SQL scripts, located in the sample's root directory (C:\SQL-NS\Samples\MusicStore):

  • CreateMusicStore.sql Creates the music store database and the tables, views, and stored procedures in it

  • PopulateMusicStore.sql Inserts some sample data into the music store

Caution

If the C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql file appears to be missing on your system (running create_music_store_database.cmd results in an error message that reads Sqlcmd: 'C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql' : Invalid filename), you probably have not completed all the setup steps given in Chapter 2. If this is the case, return to Chapter 2 and make sure you complete the steps in the "Customizing the Source Files for Your Environment" section (p. 39).

If your SQL Server is remote (not on your development machine), running create_music_store_database.cmd may fail with an error message that reads CREATE DATABASE failed. Some file names listed could not be created. Check related errors. This occurs if the paths specified in CreateMusicStore.sql for the music store database's physical files do not exist on the remote SQL Server machine. If you see this error message on your system, you will need to edit C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql. Open the file in a text editor and find the CREATE DATABASE statement near the top. Within the CREATE DATABASE statement, several file paths are specified. Notice that these all refer to directories on your development machine. Edit these paths so that they refer to valid locations for creating data and log files on your SQL Server machine. Save C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql and then run create_music_store_database.cmd again.

You can examine these SQL scripts if you're curious about the internal structure of the music store database. Notice that CreateMusicStore.sql creates all the database objects for the music store's catalog (those shown in Figure 5.3, as well as the supporting stored procedures and views) in a schema called Catalog.

As described in the previous section, the data about each song is normalized into several tables. Although this makes sense from a schema design perspective, it is inconvenient to work with from our notification application. To get all the properties of each song, we would have to join all these tables. Because it will be common to query all the song properties together, the music store database provides a view that shows all the song details in a single place. This view effectively denormalizes the song data so that it's easier to work with. Let's take a look at this view:

1.
Open Management Studio and log in to your SQL Server.

2.
Find the MusicStore database under the Databases folder in the Object Explorer and click the plus sign to expand its node.

3.
Expand the Tables node under the MusicStore database. You'll see four tables corresponding to the entities illustrated in Figure 5.3 listed (Albums, Artists, Genres, and Songs). These table names are prefaced with the schema name Catalog to isolate them from other objects we'll put in the database later.

4.
Expand the Views node under the MusicStore database and you see the SongDetails view listed (like the other objects, it's also created in the Catalog schema). This is the view that the notification application operates against.

5.
Open a new query window and issue the following query:

USE MusicStore
GO
SELECT * FROM [Catalog].[SongDetails]

This returns a set of rows showing various songs in the database. The data you see was inserted by the PopulateMusicStore.sql script you saw earlier.

Although the notification application we're going to build uses the SongDetails view, I encourage you to take a look at the other tables and stored procedures in the music store database to become familiar with its structure.


Initial Design: Seven Important Questions About Your Application

Now that you've created the model music store database, the stage is set for designing and building the notification application itself. In this section, we'll complete the first development phase: initial design. When designing any notification application, I find it helpful to answer the following seven questions about the application:

  • What does the application do?

  • Who are the target users of the application?

  • What events trigger subscription matching?

  • What kinds of subscriptions can users enter?

  • Where do the events come from?

  • What kinds of notifications does the application send?

  • What state information does the application need to keep as it runs?

Clear answers to these seven questions capture the high-level design of your application. After you answer these questions, you can move on to specifying the implementation details that relate to each one. Let's answer these seven questions for the notification application we're going to build in this chapter.

  • What does the application do?This application is a notification service for an online music store. The store provides a selection of digital music that customers can download for a fee. The notification service allows customers to request and receive notifications about new music added to the store's inventory that is of interest to them.

  • Who are the target users of the application?The application is targeted primarily at the customers of the music store. Customers wanting to receive timely notifications of new music available for purchase can set up subscriptions with the notification service. Instead of having to browse the music store repeatedly to look for new songs that have become available, the subscribers will be notified when new music that matches their stated criteria of interest is added.

  • What events trigger subscription matching?The events of interest are new songs being added to the store. When a new song is added, notifications should be generated for subscriptions that the song matches.

  • What kinds of subscriptions can users enter?Most subscribers will not want to be notified about every new song added to the store. Rather, they will want to specify which ones are of interest to them and receive notifications about only those.

    The application provides subscribers with two ways to specify their music notification preferences: They can subscribe by artist or by genre. In other words, they can choose to receive notifications about all songs by a particular artist, or all songs of a particular genre. Each subscription allows the subscriber to specify one artist or one genre, but each subscriber may have several subscriptions of each type.

    The subscriptions by artist are event-triggered. In other words, as soon as an event arrives (indicating that a new song has been added), the application evaluates the subscriptions by artist to see whether the song matches. The notifications generated for these subscriptions are relatively immediate with respect to the arrival of the event.

    In contrast, the subscriptions by genre are scheduled. Each subscription specifies a recurring schedule (for example, every week day at 9 a.m.) that will tell the application when it should be evaluated. Rather than evaluating each subscription every time a new song is added, the application evaluates the genre subscriptions at the times the subscriber has specified, against a history of songs added since the last time the subscription was evaluated. Depending on the evaluation time specified in a subscription, there may be a long delay between the arrival of a matching event and the generation of the corresponding notification.

    The SQL-NS platform has built-in support for scheduled subscriptions. Very little work is required of developers to support them in SQL-NS applications. As we prototype the application, we'll take a detailed look at this powerful platform feature.

Note

The reason that we choose to evaluate the two types of subscriptions differently has to do with selectivity, or the expected frequency of matching events. Consider a subscription by artist: Only songs by the particular artist specified in the subscription would result in a match. You would not expect many songs by any single artist to be added frequently. In contrast, subscriptions by genre are much less selective. Songs by many different artists may match a genre-based subscription. Therefore, matching events would occur much more frequently than in the case of subscriptions by artist.

If we made the subscriptions by genre event triggered, the number of notifications generated would overwhelm the subscriber. Subscribers would be bombarded with notifications throughout the dayan unpleasant user experience, to say the least. Making the subscriptions by genre scheduled allows the application to evaluate them against a whole set of events that may have accumulated and generate a set of notifications at once. The application may even group the resulting notifications and send them to the subscriber in a single, convenient message.

  • Where do the events come from?Events that indicate that new songs have been added originate from the music store catalog itself. When a new song is added to the catalog, data about the song will be submitted to the notification application.

  • What kinds of notifications does the application send?The application sends only one kind of notification: a message indicating that new songs have been added. If several such messages are going to be sent to the same subscriber (for example, if a batch of songs was added, several of which match the subscriber's subscriptions), the application combines those separate messages into a single digest message that consolidates the information from each of the separate messages.

  • What state information does the application need to keep as it runs?Because the subscriptions by genre are scheduled subscriptions, they may be evaluated well after the submission of the event data against which they need to operate. Therefore, the application must keep a history of the events it sees and make this available to the matching logic of the scheduled subscriptions.

    Each scheduled subscription should be evaluated only against the songs added since the last time it was evaluated. Because the event history contains all songs added, even those against which subscriptions have already been evaluated, the application needs to keep a log of the last evaluation time of each scheduled subscription. It can then use this log to filter out old events from the event history.

When designing any application, if you can answer the seven questions about your application to this level of detail, you probably have an initial design that's good enough to start prototyping. Obviously, the design at this point is stated only in high-level terms; it specifies almost none of the implementation details. If these are known, you could specify them in your design at this time. However, in practice, I find that it's difficult to picture all the details in advance, and even if you can, they will probably change as you implement the application. Therefore, it's best to start with a high-level design and then build a prototype to test different implementation strategies. Every real notification application I've been involved with has been built this way.


Building the Application Prototype

This chapter focuses on prototyping only the basic features of the application. These include the events, event triggered subscriptions, and notifications. You should already have a basic understanding of the concepts behind these features; the main purpose of this section is to learn incremental development and testing techniques used in the prototyping stage.

After you've learned these techniques, we'll prototype some more advanced application features in Chapter 6. These include scheduled subscriptions and the event and subscription state required to support them (as described in the initial design).

The prototype consists of the application's ADF and the supporting files required to run it. This section looks at a number of ADF elements that you already saw in Chapter 3, but we'll cover them in more detail. I'll describe each ADF element in general terms and show how it is used in the music store application. As always, this information is intended to supplement, not replace, the SQL-NS Books Online. Although I will cover most of the useful ADF elements here and in Chapter 6, I may omit some optional elements. The SQL-NS Books Online contains a complete reference to all ADF elements, and I encourage you to refer to that as you work through this book.

Building the ADF Incrementally

In the "Stages in the Application Development Process" section (p. 119), I said that a complete notification application can be quite complex and that it's a good idea to implement it in stages. The same is true for the ADF prototype itself. Instead of starting from a blank page and trying to write the whole ADF from start to finish, you can build it up incrementally, testing and refining each piece that you add as you go along. This is good development practice because it allows you to easily find and fix problems in isolation.

We'll begin with a skeleton ADF, described in the next section. We'll compile this skeleton ADF (along with a skeleton ICF) to create a new SQL-NS instance containing the outline of our application. The compilation will result in instance and application database objects that the SQL-NS engine can run against, although there won't be any real application code to execute.

Then we'll add pieces of code to the ADF and use the SQL-NS tools to compile the added code and apply the changes to the application database. At various stages, we'll start the instance and supply sample data to test its operation.

The Skeleton ADF

Listing 5.1 shows the skeleton ADF from which we'll start building our application. Although there isn't much in it, SQL-NS does consider this ADF a valid application definition and will compile it.

Listing 5.1. The Skeleton ADF for the Music Store Application

[View full width]
<?xml version="1.0" encoding="utf-8" ?>
<Application xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:xsi=http://www.w3.org/2001
  /XMLSchema-instance xmlns="http://www.microsoft.com/MicrosoftNotificationServices
  /ApplicationDefinitionFileSchema">

  <Database>
    <DatabaseName>MusicStore</DatabaseName>
    <SchemaName>SongAlerts</SchemaName>
  </Database>

  <SubscriptionClasses>
  </SubscriptionClasses>

  <NotificationClasses>
  </NotificationClasses>

  <Generator>
    <SystemName>%_NSServer_%</SystemName>
  </Generator>
  <Distributors>
    <Distributor>
      <SystemName>%_NSServer_%</SystemName>
    </Distributor>
  </Distributors>

</Application>

In this skeleton form, the application has no event, subscription, nor notification classes. It simply specifies the database and schema into which the application's database objects should be installed and defines a single generator and distributor. We'll add the event, subscription, and notification classes to the ADF in the following sections.

Note

There is a syntactical inconsistency in the ADF rules for defining event classes, subscription classes, and notification classes. An application that has no event classes must omit the <EventClasses> element entirely; an ADF that has an <EventClasses> element but no <EventClass> declarations within it is considered invalid. However, an ADF may not omit the <SubscriptionClasses> or <NotificationClasses> elements, even if it has no subscription classes or notification classes. These must be present, but they can be empty. An ADF that omits these elements is considered invalid.

Notice that the <Database> element in this ADF indicates that the application's database objects should be placed in the MusicStore database. This is the database we created and examined in the "Creating the Music Store Database" section (p. 121) earlier in this chapter. The MusicStore database already contains the tables required to store the music store's catalog and will be augmented with the SQL-NS application's database objects when we compile the ADF. If you look at the music store instance's ICF (described in the next section), you'll see that its <Database> element also specifies MusicStore as the database name. Thus, the instance, application, and music store catalog database objects will all end up in the same database, isolated from one another in separate schemas.

Though all the samples in this book place the instance and application database objects in the same database (along with other, preexisting tables and stored procedures, in this chapter's example), this doesn't have to be the case. You may deploy the instance and application database objects into any configuration of separate databases you choose, simply by specifying the appropriate database names in the <Database> element in the ICF and ADFs.

Compiling the Skeleton ADF

As you may recall from Chapter 4, all applications must be created within some instance. In this chapter, we'll create an instance called MusicStore. This instance has just a single application (the one we've been designing) that we'll build up incrementally from our skeleton ADF.

The ICF used to define the MusicStore instance is similar to the one examined in Chapter 4. If you're curious, you can take a look at it: It's called InstanceConfiguration.xml and is located in the C:\SQL-NS\Samples\MusicStore directory.

Tip

I've grouped all the files associated with the MusicStore sample into a Management Studio solution. You can find the solution file at C:\SQL-NS\Samples\MusicStore\MusicStore.ssmssln. I suggest you open this solution in Management Studio now so that you can have easy access to the various files as you work through this chapter.

Let's go through the steps necessary to create and run the instance with the skeleton ADF. These steps use the SQL-NS command-line tools to create and configure the instance and should look familiar to you after reading Chapter 4:

Caution

You must complete the steps described in the earlier section "Creating the Music Store Database" (p. 121) before proceeding with these instructions.

1.
From a Notification Services Command Prompt on your development machine, navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run create.cmd.

3.
Run register.cmd.

4.
Run grant_permissions.cmd.

5.
Run enable.cmd.

6.
Start the instance by typing the following command:

net start NS$MusicStore

If the preceding steps succeeded, the MusicStore instance should be created with our application (called SongAlerts) in it. The SQL-NS Windows service for the instance should be running, although it won't do much because we haven't implemented any of the code for our application yet.

The instance and application database objects will have been added to the existing MusicStore database by the SQL-NS compiler. To see these objects, refresh the database node in the Object Explorer tree in Management Studio. In addition to the objects in the Catalog schema that were there before, you'll see several new objects added under the Tables, Views, and Stored Procedures folders beneath the MusicStore database node.

The instance objects are prefaced with the schema name NSInstance, and the application objects with the schema name SongAlerts.

Adding Code to the ADF

The ADF for the music store application is called ApplicationDefinition.xml and is located in the C:\SQL-NS\Samples\MusicStore\SongAlerts directory. Initially, ApplicationDefinition.xml contains the skeleton ADF from Listing 5.1 that we just compiled. As we build the prototype in this chapter, we'll add pieces of code that provide parts of the application functionality to this ADF. You have two options for how you can add code to the ADF:

  • You can manually edit ApplicationDefinition.xml and type in the new code.

  • You can copy the code from a supplementary ADF that I've provided.

If you look in the C:\SQL-NS\Chapters\05\SupplementaryFiles directory, you'll see a number of files named ApplicationDefinition-0.xml, ApplicationDefinition-1.xml, and so on. The first file, ApplicationDefinition-0.xml, contains the skeleton ADF shown in Listing 5.1 (the starting point for the music store application). Each subsequent file is the same ADF with a new piece of code added to it. Each file is a valid, compilable ADF that represents our application at an intermediate stage of development. Table 5.1 lists these files, the code added in each one, and the section in this chapter that explains that code. These supplementary files are not included in the MusicStore sample's Management Studio solution.

Table 5.1. Supplementary ADFs and the Code They Contain

Filename

Code Added

Sections

ApplicationDefinition-0.xml

Skeleton ADF (as shown in Listing 5.1)

"The Skeleton ADF," p. 127.

ApplicationDefinition-1.xml

Event class and event provider

"Events," p. 135.

ApplicationDefinition-2.xml

Subscription class

"Event-Triggered Subscriptions," p. 140.

ApplicationDefinition-3.xml

Notification class

"The Output of the Application: Notifications," p. 143.

ApplicationDefinition-4.xml

Match rule for event-triggered subscriptions

"Matching Logic for Event-Triggered Subscriptions," p. 152.

As you work through this chapter, you can use the code in the supplementary ADFs listed in Table 5.1 instead of manually editing the ADF. For example, to use the code in ApplicationDefinition-1.xml, copy it over ApplicationDefinition.xml using the following commands:

[View full width]

cd /d C:\SQL-NS\Samples\MusicStore\SongAlerts
copy C:\SQL-NS\Chapters\05\SupplementaryFiles\ApplicationDefinition-1.xml
   ApplicationDefinition.xml

Note

Although there are several versions of the ADF in the files listed in Table 5.1, the ICF references only ApplicationDefinition.xml (with no numeric suffix) in the C:\SQL-NS\Samples\MusicStore\SongAlerts directory. Therefore, when you compile the instance, this is the only file that the compiler looks at. All changes you make must be made to this file.

If you're adding code by typing, make sure that you edit only ApplicationDefinition.xml. If you're using the supplementary ADFs I've provided instead of typing, make sure that you copy the supplementary ADF over ApplicationDefinition.xml. If you edit any of the supplementary ADFs (you shouldn't have to do this), your changes won't be visible to the compiler unless you copy them over to ApplicationDefinition.xml. If you mistype a copy command and accidentally overwrite the wrong file, you can always copy all the ADFs from the CD-ROM again.

It's possible, as you make the changes described in this chapter, that you might make a mistake that leaves your instance in an unusable state. If this occurs, you can revert to the skeleton ADF or one of the supplementary, intermediate ADFs provided. To revert, stop the service, run delete.cmd from C:\SQL-NS\Samples\MusicStore\Scripts, overwrite ApplicationDefinition.xml with a known good ADF, and then rerun the create instructions in the "Compiling the Skeleton ADF" section (p. 128).

Making Updates to an Instance and Its Applications

When you first create a SQL-NS instance, you pass the ICF and ADFs to the SQL-NS compiler, which builds the corresponding instance and application database objects. The ICF and ADFs are never used by the running system: The SQL-NS engine runs against the database objects only.

If you want to make changes to your instance (or any of the applications within it), you can't just change things in the database. Instead, you must use the SQL-NS tools to update the instance. To make a change, you edit the XML in the relevant ICF or ADFs and then pass the ICF to either the nscontrol update command-line tool or the Update Instance command in Management Studio. These tools compare the modified XML to the instance and application database objects that currently exist. Where they find differences, they recompile the XML and update, remove, or create the corresponding database objects. This section explains the use of the nscontrol update command and the Update Instance command in Management Studio. In keeping with the rest of this chapter, subsequent sections will refer only to the command-line nscontrol update command.

Caution

Although it sometimes may be tempting to make changes to the instance and application database objects directly (by editing a stored procedure or modifying a table schema, for example), be aware that this is extremely risky. What seems like a simple change may actually require several changes throughout the database objects to keep things consistent. If you make changes directly to SQL-NS database objects, it is easy to mess things up to the point that your instance is no longer operational. Also, if you make manual changes to the database and then use the SQL-NS tools to update the instance later, your changes will be lost. Instead of touching the database objects directly, always make changes through nscontrol update or through the Update Instance command in Management Studio.

The SQL-NS update tools can be used to perform maintenance or upgrades to a deployed instance. They can also be used to build an ADF incrementally, as we'll do in this chapter. We already compiled the skeleton ADF and, as we work through this chapter, we'll add new code to that ADF and use the SQL-NS update tools to compile and apply the changes to the database.

Caution

The SQL-NS update tools are designed to apply any changes to your databases atomically. In other words, either all the changes are made or none of the changes are made. This ensures that your instance is always left in a consistent state, even if an error occurs during the update process.

Therefore, when using the SQL-NS update tools to make changes to a production application, it's recommended that you make a backup of your databases, just in case the changes don't turn out as you intended. We won't perform those backup steps in this chapter because we're simply using the update tools to make simple additions to our application in development. If something went wrong, we could easily start over from a known good point. Chapter 13, "Deploying a SQL-NS Instance," provides more information about how to use the SQL-NS update tools safely in production environments.

Using nscontrol update

In subsequent sections in this chapter, I'll describe pieces of code that you should add to the ADF. You can add these manually or by using the supplementary ADFs I've provided, as described in the "Adding Code to the ADF" section (p. 130). At certain points, when we've added enough code to run a new piece of the application, I'll tell you to update the instance to apply the changes. To perform an update, you must first stop the instance's Windows service (NS$MusicStore) and disable the instance, then invoke nscontrol update. The following instructions describe the exact steps you will need to perform. Do not run these instructions now; I'm including them here for reference because they'll be used in many sections later in this chapter. I'll refer you back to these instructions when you need to run them.

I've provided a batch file that wraps the call to nscontrol update. This batch file is located in the C:\SQL-NS\Samples\MusicStore\Scripts directory and is called update.cmd. Take a look at the contents of this file to see how nscontrol update is called. Notice that the syntax is almost identical to nscontrol create.

1.
From a Notification Services Command Prompt on your development machine, navigate to the scripts directory for this chapter's sample by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run disable.cmd.

3.
Stop the instance by typing the following command:

net stop NS$MusicStore

4.
Run update.cmd. After nscontrol update finishes examining the instance and looking for differences between the XML and the database objects, it will print a summary of the changes it will apply and a prompt asking for confirmation. Respond with y to accept the changes.

5.
Run enable.cmd.

6.
Start the instance by typing the following command:

net start NS$MusicStore

If the preceding instructions succeeded, your instance should be running with the new changes. If you had errors in the changes you made to your instance and application, you would have seen compile errors at step 4. If this happens, fix the errors and run the update.cmd batch file again.

Using the Update Instance Command in Management Studio

To update an instance using Management Studio, you use the Update Instance command. Like nscontrol update, the Update Instance command requires you to disable the instance and stop the SQL-NS engine before it can operate. This section describes the use of the Update Instance command. Do not run the instructions described here now. Whenever you need to make updates to the instance as you work through this chapter, you can refer back to these instructions if you want to use Management Studio, instead of nscontrol update, to apply source code changes you make to the music store ADF.

The Update Instance command is launched from the context menu that appears when you right-click a Notification Services instance in the Object Explorer tree in Management Studio. The user interface of the Update Instance command is almost the same as that of the New Notification Services Instance command we looked at in the "Compiling the ICF with the Management Studio Tools" section (p. 87) in Chapter 4. The Update Instance dialog box lets you select an ICF to compile, allows you to supply values for the ICF's parameters through a parameters grid, and lets you select whether to enable the instance after the update process is complete.

Follow these instructions to update the music store instance after making changes to the ADF:

1. | Open Management Studio and connect the Object Explorer to the SQL Server hosting the MusicStore instance's database.
2. | In the Object Explorer tree, find the MusicStore instance in the Notification Services folder.
3. | Right-click the MusicStore instance and select Disable to disable the instance. Answer Yes in the confirmation dialog box.
4. | Right-click the MusicStore instance and select Stop to stop the instance's Windows service. Answer Yes in the confirmation dialog box.
5. | Right-click the MusicStore instance and select Tasks, Update.
6. | In the Update Instance dialog box, click the Browse button to browse for the music store instance's ICF. In the File Open dialog box, navigate to C:\SQL-NS\Samples\MusicStore and select the InstanceConfiguration.xml file. Make sure you open InstanceConfiguration.xml from the correct directory. If you used Management Studio to create the instances described in previous chapters, the File Open dialog box may start in another sample's directory, which may also contain an InstanceConfiguration.xml file. Caution If the C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml file appears to be missing on your system, you probably have not completed all the setup steps given in Chapter 2. If this is the case, return to Chapter 2 and make sure you complete the steps in the "Customizing the Source Files for Your Environment" section (p. 39).
7. | The parameters grid in the Update Instance dialog box will be populated with the parameters from the ICF. You should not have to change any of these values.
8. | Check the box labeled Enable Instance After It Is Updated, below the parameters grid.
9. | Click the OK button to begin updating the instance and its application. A progress dialog box will appear, listing the various steps being performed.
10. | After the first step in the progress dialog box (examining the instance) completes, an Update Summary dialog box will appear, summarizing the changes that will be made by the Update Instance command. The Update Summary dialog box consists of two panels. The top panel shows a tree that lists changes that will be made at various levels in the instance. You can select any of the items in this tree to see more details about the change in the bottom panel. After reviewing the changes in the Update Summary dialog box, click the Update button to proceed.
11. | When the remaining steps in the Update Instance progress dialog box complete, click the Close button to dismiss it.
12. | The instance and application database objects should now be updated. If you checked the Enable Instance After It Is Updated box in step 8, the instance will be enabled again. If you didn't check the box in step 8, follow the instructions in the "Enabling the Instance with the Management Studio Tools" section (p. 96) in Chapter 4 to enable the instance now.
13. | Restart the Windows service by right-clicking the MusicStore instance in Object Explorer and choosing Start from the context menu. Answer Yes to the confirmation dialog box. After the instance starts, click the Close button to dismiss the progress dialog box.

After completing these steps, your instance will be running with the changes you just made.

Implementing the Initial Design

In the initial design stage, we described the music store application in terms of its events, subscriptions, notifications, and application state. In the previous section, we created the basic skeleton of the application and looked at the development techniques we'll use to enhance that skeleton. In this section, we're going to add the code to the ADF that implements the design and test it with sample data. At the end of this section, we'll have completed the basic features in the application prototype.

Events

Conceptually, an event in our notification application is the addition of a new song to the music store. As you may recall from the discussion of the SQL-NS application model in Chapter 3, events in SQL-NS applications are modeled as data. In other words, each event is represented by a piece of data that fully describes its properties.

The event data is used to determine whether the event matches any subscriptions, and, for those subscriptions that it does match, to generate meaningful notifications. Thus, the event data must contain sufficient information to fulfill both of these functions.

Because our application supports two ways of subscribing (by artist and by genre), the event data for a given song should contain the song's artist name and genre so that we can determine which subscriptions it matches. The notification we generate for matched subscriptions describes the song added, so the event data also needs to contain other information about the song, such as the song title and the title of the album in which it belongs. At minimum, our event data would contain the following properties:

  • The artist name

  • The song genre

  • The song title

  • The album title

We could easily define an event class for our application that had fields corresponding to the preceding song properties. This would certainly be a valid way to build the application, but it may not be the most efficient.

Think about the answer to question 5 ("Where do the events come from?") in the section "Initial Design: Seven Important Questions About Your Application" (p. 124). We said that the events come from the music store catalog. The catalog tables in the music store database (described in the "Supporting the Sample Application" section, p. 120) already store all the properties of the songs that the music store knows about. If we were to make our events contain all the properties just listed, we'd just be duplicating the data that's already in the music store catalog tables. A much more efficient design would be to have our event data contain just a reference to the song data already in the music store catalog tables.

In the music store catalog, each song has a unique ID. When we looked at the SongDetails view in the music store database earlier, you may have noticed the SongId column that provides the ID for each song listed in the view. The event data needs to contain this ID because we can reference all the other properties of the song by joining with the SongDetails view, as shown in Figure 5.4.

Figure 5.4. Event data references the song properties in the music store catalog.

[View full size image]

In our application, we'll define a single event class called SongAdded. This event class will have a single field, SongId, an integer that represents the ID in the music store catalog of the song added. In this section, we'll add the code to the ADF that defines the SongAdded event class and declares the test event provider that submits events of this event class. After adding this code, we'll update the instance and then test event submission. The event class and event provider code is similar to what you saw in Chapter 3; there's not much new in this part of the application.

Declaring the Event Class

To define our application's event class, we'll add an <EventClasses> element to the ADF and insert a single <EventClass> element within it, as shown in Listing 5.2.

Note

Starting with Listing 5.2 and continuing through the rest of the listings in this chapter, I show abbreviated ADF code. My purpose in these listings is to highlight a single element or set of elements within the ADF. These parts will appear in boldface font. I show the outer enclosing <Application> element (without the XML namespace attributes) and some of the other elements in the ADF to illustrate the placement and context of the elements I'm highlighting. However, I will omit other parts of the ADF that are not relevant to the immediate discussion. Missing code will be indicated by an ellipsis (...).

This abbreviation technique is for simplicity and readability of the code listings. Most of the time, these abbreviated code listings will not compile by themselves. However, all the source files copied from the CD-ROM contain complete ADFs that will compile.

Listing 5.2. Declaration of the SongAdded Event Class

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <Schema>
        <Field>
          <FieldName>SongId</FieldName>
          <FieldType>INT</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
    </EventClass>
  </EventClasses>

  <SubscriptionClasses>
  </SubscriptionClasses>

  <NotificationClasses>
  </NotificationClasses>
  ...
</Application>

This event class declaration is similar to the one we saw in Chapter 3 for the stock application. The <EventClassName> element provides the name of the event class, and the <Schema> element provides the list of fields. The field declarations (here we have just one, the SongId) provide the names, data types, and type modifiers for the fields.

Declaring the Event Provider

Applications can have one or more event providers that submit events from the outside world. When prototyping an application, the easiest way to submit events for testing is through a T-SQL script.

In the prototyping phase, you'll often want to submit a specific pattern of events into your application to test the logic that processes events. When doing this, you want to focus on the logic itself, not on the event submission mechanism, so using a T-SQL script makes sense: All you have to do to get a specific set of events into your application is insert the appropriate data into the event class view.

Although this method is useful for testing during the prototyping phase, it is rarely used in practice as a real event provider. When you're certain that the event processing logic works, you usually replace the T-SQL script with another event provider that submits events from the real event source of your application. Chapter 8, "Events Providers," describes the built-in SQL-NS event providers you can use, as well as the mechanisms for building custom event providers.

Listing 5.3 shows the event provider declaration for our music store application's ADF. A single, nonhosted event provider is declared. When using a T-SQL script to submit events, you don't actually reference any information provided in the event provider declaration. However, the declaration is required to tell SQL-NS that events will come from a source not hosted in the SQL-NS engine. Think of this event provider declaration as a placeholder for a real event provider declaration that would provide additional configuration information.

Listing 5.3. The Event Providers Declaration in the ADF

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

    ...
  <Providers>
    <NonHostedProvider>
      <ProviderName>TestEventProvider</ProviderName>
    </NonHostedProvider>
  </Providers>

  <Generator>
    ...
  </Generator>

  <Distributors>
    ...
  </Distributors>
</Application>

The declaration just provides a name for the event provider. The fact that it is declared within a <NonHostedProvider> element tells SQL-NS that it will run external to the SQL-NS engine. In Chapter 8, you'll see an example of <HostedProvider> declarations, which configure event providers that run within the SQL-NS engine.

Testing Events

With the addition of the event class and event provider to the skeleton application, you can compile it and test event submission. You can either manually add the code described in this section to the ADF (C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml), or you can use the supplementary ADF I've provided, with the code already added. If you want to use the supplementary ADF, copy ApplicationDefinition-1.xml over ApplicationDefinition.xml as described in the "Adding Code to the ADF" section (p. 130).

Support for ADF and ICF Editing In Management Studio

If you're adding code to the ADF manually, the XML editor in Management Studio can provide syntax checking and autocompletion of XML elements as you type. This makes the process of editing an ADF much quicker and less error prone. The same support is also available when editing the ICF (which you won't do in this chapter). To enable these features, you need to load the XSD schema for the ADF (or ICF) into Management Studio. The XSD schema defines the allowed elements and syntax rules that the editor will enforce.

The XSD schemas for ADFs and ICFs are installed when you install SQL-NS. They are located in the XML Schemas folder beneath the directory in which you installed the SQL-NS files. If you installed SQL Server in the C:\Program Files\Microsoft SQL Server directory, you'll find the SQL-NS XSD schemas in the C:\Program Files\Microsoft SQL Server\90\NotificationServices\9.0.242\XML Schemas directory.

The ADF schema is in the file, ApplicationDefinitionFileSchema.xsd, and the ICF schema is in the file ConfigurationFileSchema.xsd. Open these files in Management Studio, alongside any ADF or ICF, to enable the rich XSD schema support.

When you've added the code in Listings 5.2 and 5.3 to your ADF, run the update instructions listed in the "Using nscontrol update" section (p. 132). This will compile the changes you just made and update your instance. Alternatively, if you want to use the Management Studio tools to update the instance, follow the instructions in the "Using the Update Instance Command in Management Studio" section (p. 133).

After the update completes, you can use the following steps to submit some test events:

1.
Open the C:\SQL-NS\Samples\MusicStore\MusicStore.ssmssln solution in Management Studio (if you haven't already opened it).

2.
Open the SubmitEvents.sql script. (You'll find it in the Solution Explorer, under the Queries folder in the MusicStore project.) Notice that the statements in the script simply insert various SongId values into the event class view (which has the same name as the event class, SongAdded).

3.
Run the SubmitEvents.sql script.

4.
Open a new query window and issue the following query to see the event data that was inserted into the underlying events table:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[NSSongAddedEvents]

The NSSongAddedEvents table that you just queried was created by SQL-NS when it compiled the event class declaration. For each event class, SQL-NS creates a table named NS<EventClass>Events, where <EventClass> is the name of the event class. This table has some columns that are created automatically by SQL-NS for its internal use, but also has a column for each field in the event class schema. The values for the fields declared in the event class were provided in the INSERT statement that submitted events into the events view. The values of the internal columns were generated by an insert trigger on the events view that SQL-NS created. You can explore the structure of the NSSongAddedEvents table in Management Studio to see exactly what SQL-NS produced from your event class declaration.

Event-Triggered Subscriptions

In the initial design, we said that the application allows users to subscribe for new song notifications by artist or by genre. The subscriptions by artist will be event triggered, and the subscriptions by genre will be scheduled. Each type of subscription is represented by its own subscription class.

Recall from Chapter 3 that a subscription class represents a particular type of subscription that the application supports. The subscription class definition provides a schema for the data that the application needs to keep for each subscription of that type, as well as a set of match rules used to match the subscription data with event data to form notifications. This section focuses on the schema for the subscriptions by artist, but we'll leave design of the match rules to the "Matching Logic for Event-Triggered Subscriptions" section (p. 130).

Schema for the Subscriptions by Artist

The schema of a subscription class is just a description of the data fields required to capture a subscription. A good way to come up with the schema for a subscription class is to think about how a user would phrase his subscription in words. For example, a subscription for new song notifications by artist could be phrased as

"Notify me when any song by artist A is added to the music store."

After you have the form of the subscription statement, identify the parts that can vary per subscription, and those parts become fields in your subscription schema. For new song subscriptions by artist, every subscription in the system has the same form as the preceding statement, but each subscriber could have a different value for the artist name "A."

Thus, your subscription class has a field for the artist name. Because there are no other variable parts of the subscription statement, this is the only field in the subscription class. We'll create a subscription class for the subscriptions by artists, called NewSongByArtist, with a single string field for the artist name.

Declaring the Event-Triggered Subscription Class

To declare the subscription class for our event-triggered subscriptions, we add a new <SubscriptionClass> element to the currently empty <SubscriptionClasses> element we already have in our ADF. Listing 5.4 shows the NewSongByArtist subscription class declaration.

Listing 5.4. Declaration of the NewSongByArtist Subscription Class

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>ArtistName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
    </SubscriptionClass>
  </SubscriptionClasses>

  <NotificationClasses>
  </NotificationClasses>

   ...

</Application>

As with the event class declaration, the subscription class declaration provides a name and a schema. The schema defines the fields in the subscription data. For the NewSongByArtist subscription class, there is a single string field called ArtistName. Later in this chapter, we'll add match rules to this subscription class declaration.

With the addition of this subscription class to the application, you can test the addition of some sample subscriptions. This sets the stage for testing the match rules later in this chapter. Before you can add the subscriptions, you need to compile the ADF with the subscription class declaration added. You can either manually add the code shown in Listing 5.4 to the ADF, or you can use the supplementary ADF I've provided, with the code already added. If you want to use the supplementary ADF, copy ApplicationDefinition-2.xml over ApplicationDefinition.xml, using the steps described in the "Adding Code to the ADF" section (p. 130).

When your ADF is ready, run the update instructions listed in the "Using nscontrol update" section (p. 132) or the "Using the Update Instance Command in Management Studio" section (p. 133), as you did before. This compiles the new subscription class and updates your instance accordingly.

Adding Subscriptions

The purpose of the prototype stage is to test the core functionality of the application, before the final input and output interfaces are built. Therefore, we shouldn't focus on building the full subscription management interface right now. Instead, we need to add sample subscription data in a simpler way and use this to test the processing that our application does.

As in Chapter 3, we'll use a set of T-SQL scripts that insert subscriber and subscription data. These scripts insert subscriber, subscriber device, and subscription data into the respective views in the MusicStore database. Chapter 7, "The SQL-NS Subscription Management API," covers subscribers, subscriber devices, and subscriptions in more detail and explains how to design and implement a real subscription management interface.

To invoke the scripts to add subscriber and subscription data, use the following steps (make sure that you have successfully updated your application with the subscription class code shown in the previous section before running these instructions):

1. | In the MusicStore Management Studio solution, open the files AddSubscribers.sql and AddSubscriptions.sql from the Queries folder of the MusicStore project in the Solution Explorer.
2. | Execute the AddSubscribers.sql script to add the subscriber data. It's important that you do this before running AddSubscriptions.sql because the subscriber records must be in place before you can add subscriptions.
3. | Execute AddSubscriptions.sql.

Let's take a look inside the database and observe the changes made by updating the application with the new subscription class and adding the subscription data.

1.
In Management Studio's Object Explorer, right-click the MusicStore database in the Databases folder (note: not the MusicStore instance in the Notification Services folder) and select Refresh. This updates the Object Explorer's view of the objects in the database (which have changed because we updated the SQL-NS instance).

2.
Expand the MusicStore database node and the Tables folder beneath it.

3.
Look for a table named SongAlerts.NSNewSongByArtistSubscriptions.

This is the table in which subscriptions data for the NewSongByArtist subscription class is stored. When processing the ADF, the SQL-NS compiler creates a table for each subscription class, named NS<SubscriptionClass>Subscriptions, where <SubscriptionClass> is the name of the subscription class.

4.
Expand the SongAlerts.NSNewSongByArtistSubscriptions node and the Columns node below it.

There should be six columns: The first five are created automatically by SQL-NS, but the last one comes from the single field in our subscription class, ArtistName. Had our subscription class had more fields, they would have appeared as additional columns here.

5.
Open a new query window and issue the following query:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[NSNewSongByArtistSubscriptions]

This returns a set of rows that correspond to the subscriptions just added.

The Output of the Application: Notifications

At this point, we've prototyped the inputs to the application: events and subscriptions. It's now time to design the application's output: the notifications it generates and sends.

Note

It might seem logical to design the matching logic at this time because we already have the events and subscriptions implemented. But we must design the notification classes of the application first because the matching logic must create notifications of a particular notification class. Unless the notification classes are defined first, it's not possible to write the matching logic.

Just as events are described by event classes, and subscriptions are described by subscription classes, notifications are described by notification classes. Each notification class encapsulates the following three aspects of the notifications it represents:

  • The data that the notification contains

  • The formatting of the final notification message

  • The means by which the notification gets to its recipient

This section looks at the ADF elements used to specify this information.

In our music store application, we have only one type of notification: a message indicating that a new song was added. Thus, we have only one notification class, and we will call it NewSong. In this section, we'll implement the NewSong notification class in the ADF.

Notification Schemas

The schema of a notification class defines the data that each notification contains. When defining a notification class schema, it's important to think about what a notification message should look like when presented to the subscriber in its final form. The notification class should be designed so that it contains sufficient data to construct that notification message.

The purpose of the NewSong notifications in our application is to inform subscribers of new songs added to the database. To be useful, each notification should include a complete description of the song. As we saw when we examined the music store database earlier in this chapter (see "The Music Store Database Schema," p. 121), the music store keeps the following properties of each song:

  • Song title

  • Artist name

  • Album title

  • Genre

To give the subscriber as much information as possible, each NewSong notification should contain all these properties. Each one becomes a string field in the notification class schema. Listing 5.5 shows the first part of the NewSong notification class declaration in the ADF. (Listings 5.6 through 5.8 show the remaining parts of the notification class declaration.)

Listing 5.5. Declaration of the NewSong Notification Class

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

  ...

  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      <Schema>
        <Fields>
          <Field>
            <FieldName>SongTitle</FieldName>
            <FieldType>NVARCHAR(255)</FieldType>
        </Field>
        <Field>
          <FieldName>ArtistName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
        </Field>
        <Field>
          <FieldName>AlbumTitle</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
        </Field>
          <Field>
            <FieldName>Genre</FieldName>
            <FieldType>NVARCHAR(255)</FieldType>
          </Field>
        </Fields>
      </Schema>
      ...
    </NotificationClass>
  </NotificationClasses>

  ...

</Application>

We've added a new <NotificationClass> element in the ADF's <NotificationClasses> element that was previously empty. The declaration provides the name of the notification class in the <NotificationClassName> element and its schema in the <Schema> element. Looking at the declaration, you see that a string field is declared for each of the song properties listed previously.

Notification Formatting

The notification schema just provides a description of the raw data that the notification will contain. But before a notification can be delivered to a subscriber, it has to be formatted. In the formatting process, the raw data is manipulated and decorated so as to make it presentable to the final recipient. In this section, we'll look at how the notification formatting options are declared in the notification class.

SQL-NS employs components called content formatters to format raw notification data after it is generated. Content formatters are passed the raw notification data as input; they process this data and return a string that represents the formatted notification as output. SQL-NS ships with a built-in content formatter that uses XSL transforms to format notification data (we saw this in use in Chapter 3). You can also build a custom content formatter for your application, as described in Chapter 9, "Content Formatters." Each notification class contains a declaration that tells SQL-NS which content formatter to use when formatting its notifications.

Although notification formatting is an important aspect of any application, it doesn't deserve much attention at the prototype stage. As mentioned previously, the prototype stage is about developing the core application functionality; the form of the final input and output should be designed later. Therefore, instead of trying to build a content formatter now, we will use the built-in XsltFormatter to quickly test the notifications aspect of our application.

Note

Although used here for testing our prototype, the XsltFormatter is actually a powerful component often used to do the real formatting work in production SQL-NS applications. Chapter 9 describes the details of the XsltFormatter and shows you how to customize it for your own applications.

Listing 5.6 shows the part of the NewSong notification class declaration that specifies the content formatter.

Listing 5.6. Declaration of the Content Formatter in the NewSong Notification Class

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

  ...

  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      <Schema>
        ...
      </Schema>
      <ContentFormatter>
        <ClassName>XsltFormatter</ClassName>
        <Arguments>
          <Argument>
            <Name>XsltBaseDirectoryPath</Name>
            <Value>%_ApplicationBaseDirectoryPath_%\XslTransforms</Value>
          </Argument>
          <Argument>
            <Name>XsltFileName</Name>
            <Value>NewSong.xslt</Value>
          </Argument>
        </Arguments>
      </ContentFormatter>
      ...
    </NotificationClass>
  </NotificationClasses>

  ...

</Application>

The value of the <ClassName> element tells SQL-NS which content formatter to use. In this case, it specifies the XsltFormatter. The remaining elements in the <ContentFormatter> declaration specify arguments for the content formatter components. The number, type, and purpose of the arguments vary for each formatter. The XsltFormatter requires two arguments:

  • XsltBaseDirectoryPathSpecifies the base directory path in which the XSL transform files can be found. (As will be described in detail in Chapter 9, there can be multiple XSL transform files in subdirectories off this base directory that provide different formatting for different languages and device types; in this chapter, we have just one file.)

  • XsltFileNameSpecifies the name of the XSL transform file.

Note

If you're curious, you can look at the XSL transform we'll use in the application prototype: C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\NewSong.xslt. However, I'm not going to explain the format of this file or how it is used by the XsltFormatter here; Chapter 9 covers those topics in detail.

Aggregation Options: Digest and Multicast

SQL-NS provides two useful delivery features not discussed so far: digest and multicast. The digest feature allows you to group several individual notifications all going to the same subscriber into a single message. Multicast allows you to send the same message to many different subscribers.

Digesting is useful when it's likely that a single batch of events will generate more than one notification for a given subscriber. In our music store application, this is certainly the case. Imagine the store adding a whole new album to its inventory. The album would contain several songs by the same artist, and any NewSongByArtist subscription that matched would result in several notifications, one for each song.

These notifications can always be sent individually to the subscriber, but doing so would result in a flurry of separate messages that would be difficult (or annoying) for the subscriber to manage. The subscriber probably wants to see the information about all the songs at once anyway, so it makes sense to group these individual notifications into a single message. Digesting allows you to do this.

Multicast can be thought of as the inverse of digest: Where digest groups several notifications for one subscriber into a single message, multicast allows you to send a single notification to multiple subscribers. Multicast is really a performance optimization for cases where you have the same notification going to many recipients. Instead of incurring the cost of formatting the same notification data over and over for each individual subscriber, multicast allows you to format the data once and send that single formatted message to each subscriber who requested the notification. Unlike digest, which actually affects the content that the subscriber sees, multicast is not visible to the subscriber. Subscribers receive the same notifications they would have received had multicast delivery not been used. The difference is apparent only as a reduction in the workload on the server that formats and delivers notifications.

Caution

Multicast delivery is available only on Enterprise Edition of SQL-NS. If you are using Standard Edition, you cannot use multicast delivery.

Due to limitations in the SQL-NS engine, digest and multicast cannot be used together in the same notification class. Therefore, you have to choose which is the more appropriate to use in your application. A number of factors affect this decision, but the following general guidelines usually apply:

  • Use digest delivery if a single event batch is likely to generate multiple notifications for a single subscriber and it makes sense for those notifications to be delivered together.

  • Use multicast delivery if your application generates the same notification for many subscribers.

In some applications (our music store application is an example), both of these conditional guidelines are satisfied. In cases like this, you have to trade off the end-user benefit of digest delivery against the performance gains from multicast delivery. Chapter 12, "Performance Tuning," discusses performance issues and the use of multicast delivery in more detail.

In our application, we will use digest delivery. Listing 5.7 shows the code we need to add to our notification class to enable digest delivery.

Listing 5.7. Aggregation Options in the NewSong Notification Class

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <ContentFormatter>
        ...
      </ContentFormatter>
      <DigestDelivery>true</DigestDelivery>
      <MulticastDelivery>false</MulticastDelivery>
      ...
    </NotificationClass>
  </NotificationClasses>

  ...

</Application>

The <DigestDelivery> element is given the value "true," indicating that digest delivery should be used. The <MulticastDelivery> element specifies "false," indicating that multicast delivery should not be used. Omitting one or both of these elements in the notification class is the same as specifying the value "false." If you specify "true" for both <DigestDelivery> and <MulticastDelivery>, you get an error when you compile the ADF. You also get a compile error if you specify "true" for <MulticastDelivery> and you're using the Standard Edition of SQL-NS.

Notification Delivery

Thus far, the elements of the notification class we've defined have dealt with the structure of the notification data and how that data is transformed into a message suitable for the final recipient. However, we have not described how that message actually gets to its final recipient.

As we saw in Chapter 4, a SQL-NS instance defines one or more delivery channels used to send notifications. Each delivery channel represents a delivery endpoint used to route notifications to subscribers. Each delivery channel uses a particular delivery protocol to exchange messages. In the notification class declaration in the ADF, you list the protocols that your notification class supports.

SQL-NS ships with a built-in delivery protocol that writes notifications to a file (instead of really delivering them over a network). This delivery protocol is rarely used in production systems but makes for a handy way to test your application in the prototype stage: You can open the file that this protocol writes to and examine the output of your application. Listing 5.8 shows the ADF elements that declare support for the File delivery protocol in the NewSong notification class.

Listing 5.8. Declaration of Protocol Support in the NewSong Notification Class

<Application>
  ...
  <EventClasses>
    ...
  </EventClasses>

  ...

  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <DigestDelivery>true</DigestDelivery>
      <MulticastDelivery>false</MulticastDelivery>
      <Protocols>
        <Protocol>
          <ProtocolName>File</ProtocolName>
        </Protocol>
      </Protocols>
    </NotificationClass>
  </NotificationClasses>

  ...

    ...

</Application>

The <Protocols> element contains the list of supported protocols, each of which appears in a <Protocol> element. Here, there is a single <Protocol> element for the File delivery protocol. As you'll see in Chapter 10, "Delivery Protocols," some delivery protocols require additional information in the <Protocol> element (the File delivery protocol just requires you to declare the protocol's name). In Chapter 10, we'll look at other protocols that SQL-NS provides, as well as techniques for building custom delivery protocols for your application.

Compiling the Notification Class

Together, the code fragments in Listings 5.5, 5.6, 5.7, and 5.8 form a complete notification class declaration that you can compile and test. You can either manually add the code shown in these listings to the ADF, or you can use the supplementary ADF with the code already added, ApplicationDefinition-3.xml, as described in the "Adding Code to the ADF" section (p. 130). When your ADF is ready, run the update instructions listed in the "Using nscontrol update" section (p. 132) or the "Using the Update Instance Command in Management Studio" section (p. 133). This compiles the new notification class and updates your instance.

When the update is complete, you can take a look inside the database to see the objects the compiler created for the notification class. The following steps highlight some of these database objects:

1. | In Management Studio, refresh the MusicStore database in the Object Explorer tree.
2. | Expand the MusicStore database node and the Tables node beneath it. Look for a table named SongAlerts.NSNewSongNotifications. This is the table in which the raw data that represents NewSong notifications is stored. As we saw in Chapter 3, SQL-NS creates a table for each notification class, named NS<NotificationClass>Notifications, where <NotificationClass> is the name of the notification class.
3. | Expand the SongAlerts.NSNewSongNotifications table node and the Columns node below it. Several of the columns in the list were created by SQL-NS for its internal use. But also, in the midst of the column list, you should see a column for each of the fields of the notification class: SongTitle, ArtistName, AlbumTitle, and Genre.

At this point, no data is in the notification table because no notifications have been generated. When we add the matching logic to the application and test it with sample data, a row will appear in this table for each notification generated.

In addition to the notifications table, the other important object created by the SQL-NS compiler is the notifications view. As you saw in Chapter 3, match rules insert data into the notifications view to generate notifications. The view always has the same name as the notification class. Let's take a look at the view for the NewSong notification class in Management Studio:

  1. Expand the Views node under the MusicStore database in the Object Explorer.

  2. Find the SongAlerts.NewSong view. Expand its node and the Columns node beneath it.

    Notice that the columns in the view are a subset of the columns in the notifications table, including the fields of the notification class: SongTitle, ArtistName, AlbumTitle, and Genre. Some of the other columns on the notifications table, such as those used to track delivery status, are not included in the view.

When the matching logic of the application generates new notifications of the NewSong notification class, instead of inserting rows directly into the notifications table, it should insert into the notifications view. A trigger on the view inserts the notification data into the notification table and sets up all the internal state information required by SQL-NS to process the notifications. We'll see this view used in the section "Matching Logic for Event-Triggered Subscriptions"(p. 152).

In addition to the notifications table and view just described, several other views and stored procedures were created as a result of the notification class declaration being compiled. You can explore these in the database if you choose (many will have the name of the notification class NewSong in some part of their names), but be aware that most of these are used internally by SQL-NS, and it's unlikely that you'll ever need to deal with them directly.

Matching Logic for Event-Triggered Subscriptions

With an event, subscription, and notification class defined in our music store application, we can start to implement the matching logic. At this stage, we've only created the event-triggered subscription class, so we will only implement the event-triggered matching logic in this section. When we implement the scheduled subscription class in Chapter 6, we'll implement the corresponding matching logic for its subscriptions.

Event-triggered matching logic specifies how to match subscriptions of a particular subscription class with events of a particular event class. The matching logic is always declared as part of the subscription class in the ADF, and its declaration specifies which event class it applies to. As you may recall from Chapter 3, developer-defined matching logic (the only kind of matching logic we use in the music store application) takes the form of SQL statements that join the data in the event and subscription tables to produce a set of rows that end up in the notifications table of some notification class. These SQL statements are called match rules.

When designing the matching logic for a subscription class, follow these steps:

1. | Decide which event class is associated with the matching logic (sometimes referred to as the triggering event class).
2. | Decide which notification class the resulting notifications belong to.
3. | Imagine the events and subscriptions as rows in tables whose schemas are those defined in the respective event and subscription classes. Imagine that notifications will be rows in a table with the schema defined in the notification class.
4. | Write the SQL statement that joins the events and subscriptions table and returns only rows that represent events and subscriptions that match. The columns in the resultset must match the schema of the notification class.

Let's use these steps to design the event-triggered matching logic for the NewSongByArtist subscription class in our music store application.

First, the triggering event class is SongAdded: the event class that represents new songs added to the music store database. The notification class is NewSong: the one that represents a notification about a new song.

Now, let's picture the event and subscription data as rows in tables. According to the event class definition (refer to Listing 5.2, p. 137), the events table will have a single column, SongId. The subscriptions table will also have a single column, ArtistName, according to the subscription class schema defined in Listing 5.4 (p. 141).

The result of matching the events and subscriptions will be a set of NewSong notifications. Think of these as rows in a table with the schema of the NewSong notification class, as defined in Listing 5.5 (p. 144). That table will have SongTitle, ArtistName, AlbumTitle, and Genre columns.

After we've established the schemas of the tables we're working with, we can easily write the appropriate SQL join. What we're really trying to do is join the events with the subscriptions on the artist name. However, the events don't contain the actual properties of the songs (such as the artist name); the events contain just the song IDs. To get the song properties, we have to join the events table with the SongDetails view in the MusicStore database's Catalog schema that we looked at earlier (see "Creating the Music Store Database," p. 121). Then we can join this with the subscriptions table on artist name. In SQL terms, we could write this join as follows:

SELECT  songs.SongTitle,
        songs.ArtistName,
        songs.AlbumTitle,
        songs.GenreName
FROM    [SongAlerts].[SongAdded] events
        JOIN [Catalog].[SongDetails] songs ON events.SongId = songs.SongId
        JOIN [SongAlerts].[NewSongByArtist] subscriptions ON
               subscriptions.ArtistName = songs.ArtistName

Caution

Do not actually run this query (it won't work because some of the entities it references do not yet exist). It's just here to illustrate the concept of matching the events and subscriptions using a join.

Note that this is not the final match rule that we will put in our ADF; this is a simplified version of the join that I'm showing here for illustration. As written, the statement selects the raw data that forms the content of the notification (the song properties). We'll need to select other properties, such as the subscriber ID (so that we know which subscriber should receive the notification), in the final version match rule.

As we said earlier, the output of the match rule is a set of notifications of the NewSong notification class. So, the rows returned by the previous join statement need to end up in the notifications table for the NewSong notifications class. When we compiled the notification class in the previous section, we saw that the compiler generated a view into which the rows should be inserted. Triggers on the view insert the appropriate data into the notifications table. We can simply insert the results of the match rule's SELECT clause into the notifications view.

Caution

Never insert rows directly into the notifications table. Instead, always insert into the notifications view to get the data into the notifications table and the required state set up. If you insert directly into the notifications table instead of using the view, the notifications will not be processed.

We can rewrite the match rule using the notifications view as follows:

INSERT INTO [SongAlerts].[NewSong]
SELECT subscriptions.SubscriberId,
       N'DefaultDevice',
       N'en-US',
       songs.SongTitle,
       songs.ArtistName,
       songs.AlbumTitle,
       songs.GenreName
FROM   [SongAlerts].[SongAdded] events
JOIN   [Catalog].[SongDetails] songs
    ON events.SongId = songs.SongId
JOIN   [SongAlerts].[NewSongByArtist] subscriptions
    ON subscriptions.ArtistName = songs.ArtistName

Note

Although this is the final form of the match rule, you won't be able to run this directly in Management Studio right now because it relies on views of the event and subscription data that SQL-NS populates only at runtime. These views are described later in this section. Chapter 11, "Debugging Notification Generation," describes ways to manually run match rule queries outside the SQL-NS engine.

Note that in the final version of the match rule, we've added a few more columns to the select list so that we can supply values for all the columns in the notifications view. As you may recall from Chapter 3, the first three columns in the notifications view are always the subscriber ID of the intended notification recipient, the subscriber device name, and the locale name. Here we've obtained the subscriber ID from the subscriptions and hard-coded values for the subscriber device name and locale.

Note

We'll look at subscriber devices in more detail in Chapters 7 and 10, and locales in Chapter 9.

In the FROM clause, we have the events, joined with the song details from the music store catalog, and the subscriptions. The event and subscription data come from views that SQL-NS populates immediately before running the match rule. SQL-NS creates a view with the same name as the event class, SongAdded in this case, that contains just the events that should be processed by the match rule. Similarly, it creates a view over the subscriptions table (named the same as the subscription class, NewSongByArtist) that contains the subscription data against which the rule should be evaluated. The columns in these views correspond to the schema of the respective event and subscription classes. Additionally, the subscriptions view contains a column for the subscriber ID.

Declaring Match Rules in the ADF

Listing 5.9 shows how the event-triggered match rule is declared in the ADF, as part of the subscription class declaration.

Listing 5.9. Declaration of the Event-Triggered Match Rule

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      <Schema>
        ...
      </Schema>
      <EventRules>
        <EventRule>
          <RuleName>MatchNewSongsByArtist</RuleName>
          <Action>
            INSERT INTO [SongAlerts].[NewSong]
            SELECT subscriptions.SubscriberId,
                      N'DefaultDevice',
                      N'en-US',
                      songs.SongTitle,
                      songs.ArtistName,
                      songs.AlbumTitle,
                      songs.GenreName
            FROM   [SongAlerts].[SongAdded] events
            JOIN   [Catalog].[SongDetails] songs
                ON events.SongId = songs.SongId
            JOIN   [SongAlerts].[NewSongByArtist] subscriptions
                ON subscriptions.ArtistName = songs.ArtistName
          </Action>
          <EventClassName>SongAdded</EventClassName>
        </EventRule>
      </EventRules>
    </SubscriptionClass>
   </SubscriptionClasses>
   ...

</Application>

The subscription class has an <EventRules> element that contains an <EventRule> element for each match rule. The <EventRule> element specifies a name for the rule, an <Action> element that contains the SQL implementation of the rule, and an <EventClassName> element that specifies which event class should trigger the rule's evaluation.

With the matching logic in place, we should also add some configuration options that control the frequency at which SQL-NS engine executes the match rules and the postprocessing that formats and delivers the resulting notifications. These are shown in Listing 5.10:

Listing 5.10. Configuring Generator and Distributor Quantum Durations

<Application>
  ...
  <Distributors>
    <Distributor>
      <SystemName>%_NSServer_%</SystemName>
      <QuantumDuration>PT15S</QuantumDuration>
    </Distributor>
  </Distributors>

  <ApplicationExecutionSettings>
    <QuantumDuration>PT15S</QuantumDuration>
  </ApplicationExecutionSettings>

   ...

</Application>

The <QuantumDuration> element added to the existing <Distributor> element specifies how often the distributor component in the SQL-NS engine (responsible for formatting and delivery operations) should look for new notification batches to process. Here we've set the frequency to once every 15 seconds, using the XSD duration syntax (described in detail in the sidebar, "The XSD duration Data Type", p. 157).

The <ApplicationExecutionSettings> element added to the ADF specifies another <QuantumDuration> element that applies to the generator. The value of this element controls how often the generator looks for new event batches that it needs to process using the application's match rules. The value specified here configures the generator to look for new event batches once every 15 seconds.

Had we not specified quantum durations for the distributor and generator, the default value of one minute would have been used for both settings. Though using this default would not have affected the functionality of the application, it would have meant we'd have to wait longer for the application to process events that we submit and the notifications that the match rule generates. During testing, it's convenient to use relatively short quantum durations to make the application more responsive. However, in production, longer quantum durations may be appropriate. Chapter 12 describes the performance tradeoffs involved in adjusting generator and distributor quantum durations and offers guidance on choosing the most appropriate values for your applications.

The XSD Duration Data Type

Values that represent periods of time are usually stated in the syntax of the XSD duration data type in SQL-NS ADF and ICF elements. In this syntax, time interval values are expressed in the following format:

P<years>Y<months>M<days>DT<hours>H<minutes>M<seconds>S

The "P" stands for "period" and is always required. <years>, <months>, and <days> represent the number of years, months, and days, respectively. "T" is the date and time separator and, like the initial "P", is always required. <hours>, <minutes>, and <seconds> represent the number of hours, minutes, and seconds, respectively. For example, consider an interval value specified as

P0DT00H00M30S

This specifies a period of 30 seconds (the days, hours, and minutes are zero). The year and month components are omitted entirely (it's legal to omit components that have zero values). For more information on the XML duration data type, refer to the XML schema reference at http://www.w3.org/TR/xmlschema-2/#duration.

Add the code shown in Listings 5.9 and 5.10 to your ADF, either by typing it in manually or using the supplementary ADF, ApplicationDefinition-4.xml, as described in the "Adding Code to the ADF" section (p. 130). After your ADF is ready, run the update instructions as you've done before.

Testing the Event-Triggered Matching Logic

After the application is updated with the matching logic and configuration settings, we can perform a test to show that it works. Earlier, we tested the events and subscriptions independently; now we can test them together, using the matching logic to produce notifications.

Recall that in the "Events" section (p. 135), we tested our event class by inserting event data via a T-SQL script. At the time we did that, no subscriptions were in the application (we did not even have the subscription class defined), so when those events were processed, nothing happened. Now that we have subscriptions (added in the "Event-Triggered Subscriptions" section, p. 140) and match logic, we need to submit another batch of events and see them get processed. The old batch of events we previously submitted will do us no good because they've already been processed: SQL-NS only processes each batch of events once.

Caution

Before running these instructions, make sure that the instance's Windows service (NS$MusicStore) is running. Also, if for any reason you had to re-create your instance from scratch (instead of doing an update), make sure that you've added the subscribers and subscriptions as described in the "Adding Subscriptions" section (p. 142). If you don't have any subscribers and subscriptions in your instance, you will not see notifications generated.

1. | In the MusicStore Management Studio solution, use the Solution Explorer to open the SubmitEvents.sql script (found under the Queries folder in the MusicStore project). Do not run the script yet.
2. | Open the FileNotifications.txt file, located in the Miscellaneous folder of the MusicStore project. This is the file to which the delivery protocol we configured in our application will write the notification messages. The file should be blank when you first open it.
3. | Run SubmitEvents.sql.
4. | Wait about 1 minute. This is necessary because the generator is configured to look for new event batches every 15 seconds. Depending on when in the generator's processing cycle you submit the events, it may take up to 30 seconds for the events to be processed. After the notifications are generated, there may be another delay of up to 30 seconds before they are formatted and delivered (because we also set the distributor's quantum duration to 15 seconds).
5. | Management Studio should eventually pop up a dialog box telling you that the FileNotifications.txt file has changed and asking you if you want to reload it. Choose Yes to reload the file. The file now contains the formatted notifications "delivered" by the File delivery protocol.

At some time during the interval that you waited in step 4, the generator populated the events view with the event data you submitted, ran the match rule against it, and generated a set of notifications. If you look in the notifications table (SongAlerts.NSNewSongNotifications) in your application database, you'll see the raw notification data. Compare the data in the notifications table with the event data (in SongAlerts.NSSongAddedEvents) and subscription data (in SongAlerts.NSNewSongByArtistSubscriptions) and verify that the right matches were generated. You should see notifications only for events that referenced songs in which the artist name matched the artist name specified in one or more subscriptions.

After the notifications were generated, the distributor formatted them with the XsltFormatter and delivered them through the File delivery protocol. This resulted in the output file contents you examined in step 5.

A sample notification in the output file might look like this:

Notification Id: 1
Notification Class Name: NewSong
Subscriber Id: Alex
Device Address:
Protocol Fields:

Body:
New songs are available for download from the music store!

        Song Title:  White Flag
        Artist Name: Dido
        Album Title: Life for Rent
        Genre:       Rock

The first part of the output lists the notification ID, notification class name, subscriber ID, device address, and protocol fields. (We'll take a look at device addresses and protocol fields in Chapter 10.) The File delivery protocol always writes these values as standard headers for each notification. The next part is the formatted body of the notification (the output from the content formatter). When you look at the output file, notice that some of the notifications contain information about more than one song. This is because of the digesting option set on the notification class: Separate notifications going to the same subscriber were combined into a single digest message.

Note

Although you probably experienced a noticeable delay in the processing of the events and notifications as you performed the instructions in this section, it's important to understand that the actual work done by the SQL-NS engine consumed only a tiny fraction of the full amount of time that you waited. The delays you saw were because the generator and distributor are configured to poll for new work at specified intervals. As you saw in Listing 5.10, we're using polling intervals of 15 seconds in this chapter. During most of the time in those intervals (the majority of the time you waited), the engine was simply idle.

In real applications, the use of the polling intervals allows several event and notification batches to accumulate and be processed together. This results in overall efficiency gains, but it does mean that there is a latency in the SQL-NS pipeline. You can adjust the polling intervals to change this latency as appropriate for your application. We'll look at how to do this in Chapter 12.


Summary

A SQL-NS application consists of several functional components. Some of these components, including the event providers and the subscription management interface, deal with getting data into the application. Other components, such as the content formatters and delivery protocols, deal with the output of the application. At the core of the application is a matching component that processes the input data and produces the final notifications as output. In this matching component, the core logic of the application is implemented.

The most effective way to build an application is to begin by designing and prototyping the matching component. In the design stage, you define the key elements of the application. In the prototype stage, you implement the design and test it with sample data. SQL-NS provides facilities you can use to get data in and out of the application before you build the real input and output components your application will use in its final form. Using the components provided by SQL-NS at the prototype stage allows you to focus on the implementation of the core logic and not be distracted by the mechanics of the application's input and output interfaces.

Building the prototype involves writing the application's ADF. The ADF defines the schema of the input and output data and the logic used to generate notifications. Rather than trying to build the whole ADF all at once, it's much easier to build it in stages. Starting from a skeleton ADF, you can add one piece of functionality at a time and compile it incrementally with the SQL-NS update tools. As you add each piece of functionality, you can test it in isolation. When the prototype is complete, it's a good idea to recompile the whole application from scratch and test it in an end-to-end scenario.

In this chapter, we designed and prototyped the basic features of a sample application: a notification service for a music store. In the upcoming chapters, we'll continue the development process on the music store application.

Note

In Chapter 6, we will continue to work with the instance created in this chapter. For this reason, do not delete this chapter's instance as you've done in other chapters. I've provided full cleanup instructions at various points later in this book for use when you're finished working with the instance.


Chapter 6. Completing the Application Prototype: Scheduled Subscriptions and Application State

IN THIS CHAPTER

  • Scheduled Subscriptions in the Music Store Application

  • Source Code for This Chapter

  • Event State

  • The Scheduled Subscription Class

  • Testing Scheduled Subscriptions

  • Final Testing of the Prototype

In Chapter 5, "Designing and Prototyping an Application," we designed the music store application and prototyped its basic features. We used incremental development and testing techniques to build the application one piece at a time and ensure that each piece works as expected. In this chapter, we continue using those techniques to prototype some of the more advanced features of the application's matching component: scheduled subscriptions and state maintenance. After implementing these features, the prototype will be complete and ready for end-to-end testing.


Scheduled Subscriptions in the Music Store Application

At this point, of the two kinds of subscriptions that the music store application is supposed to support, we've implemented only one: subscriptions by artist name. The initial design said that the application will also support subscriptions by genre. Using this second kind of subscription, subscribers interested in a particular genrefor example, jazzcan request to be notified whenever any song of that genre is added to the music store.

Because subscriptions by genre will be matched by many more songs than the subscriptions by artist (genre is a less-selective criterion than artist name), we said that the genre subscriptions should be scheduled, rather than event triggered. Each subscription will specify a recurring schedule on which it should be evaluated. For example, a subscription might say, "Notify me at 5:00 p.m. every weekday about any new jazz songs added to the music store." Rather than evaluate the subscription whenever new events arrive, SQL-NS will evaluate it only on the schedule specified.

In this chapter, we implement the music store application's subscriptions by genre, using the built-in SQL-NS support for scheduled subscriptions. Before defining the actual subscription class, we have to modify the application to maintain the state information that the scheduled subscription matching logic requires. Then we'll implement the scheduled subscription class schema and match rule and test the application with sample data.


Source Code for This Chapter

The material in this chapter builds on the prototype code developed in Chapter 5. It assumes you have completed all the instructions in Chapter 5 and that everything up to the event-triggered matching logic works as expected. At this point, your ADF in the music store application's source directory, C:\SQL-NS\Samples\MusicStore\SongAlerts, should contain the same code as the final supplementary ADF, ApplicationDefinition-4.xml, in C:\SQL-NS\Chapters\05\SupplementaryFiles.

Just as in Chapter 5, I've provided supplementary ADFs that contain the new code shown in this chapter. As you go through the code listings, you can either type the new code manually or use these supplementary ADFs. Table 6.1 lists the supplementary ADFs for this chapter. You'll find these files in this chapter's source directory, C:\SQL-NS\Chapters\06\SupplementaryFiles.

Table 6.1. Supplementary ADFs and the Code They Contain

Filename

Code Added

Sections

ApplicationDefinition-5.xml

Event chronicle and event chronicle rule.

"Event State," p. 163.

ApplicationDefinition-6.xml

Scheduled subscription class, subscription chronicle, scheduled match rule, and chronicle rule. (This is the final ADF.)

"The Scheduled Subscription Class," p.168.

The instructions provided in Chapter 5 for adding code to the ADF and then updating the music store instance still apply. You should already be familiar with these instructions from the work you did in Chapter 5. The only difference is that if you're using the supplementary ADFs, you'll copy them from the Chapter 6 supplementary files directory C:(\SQL-NS\Chapters\06\SupplementaryFiles) instead of the Chapter 5 supplementary files directory. You will still be copying these files to the same location, overwriting the music store application's ADF at C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml. For reference, you can find the full instructions in the "Adding Code to the ADF" section (p. 130) and "Making Updates to an Instance and Its Applications" section (p. 131) of Chapter 5.


Event State

The logic implemented in the music store application so far is completely stateless. When events arrive, they get evaluated against the subscriptions, and notifications are generated. Nothing about the processing of the events and subscriptions affects the processing of future events and subscriptions because the application maintains no memory of its past work.

Note

SQL-NS does keep track of which events have been processed (so that they are not processed again), but this is not part of the application logic.

Adding scheduled subscriptions requires the application to maintain state. Because the scheduled subscriptions are not necessarily evaluated at the time that events arrive, the application has to "remember" all the events it sees so that when the scheduled subscriptions are evaluated, they have the appropriate data to work against. In this section, we'll build the logic necessary to maintain event history. Without this, after events are matched against the event-triggered subscriptions, they are effectively forgotten. Although the event data is persisted in the events table, it isn't made available to the matching logic again, and it could be garbage collected at any time.

Event Chronicles

SQL-NS allows applications to maintain state in chronicles. A chronicle is a table (or set of tables) defined by an application to hold its state data. SQL-NS provides places in the ADF for you to declare chronicle tables and a set of chronicle rules that run at specific times to keep the chronicles up-to-date.

SQL-NS supports chronicles of two types: event chronicles and subscription chronicles. This section describes the event chronicles required to keep the event history for the music store application. The "Subscription State" section (p. 170) covers subscription chronicles.

Event chronicles are declared as part of an event class, and the chronicle rules are run whenever new events of that event class are submitted. When designing an event chronicle, you must decide on the schema of the chronicle tables and the logic of the chronicle rules that maintain the data in those tables.

The purpose of the event chronicle in the music store application is to keep a history, or log, of all the events seen. Each entry in the log represents a single event and must contain all the data in the event. Thus, the chronicle table needs a column for each field in the event class schema. Also, the chronicle table should keep a time stamp for each event, indicating when it was added to the chronicle. This time stamp allows the matching logic for the scheduled subscriptions to distinguish old events it has already seen from new events it has yet to process.

Declaring Event Chronicles

In the SongAdded event class declaration, we declare an event chronicle called SongAddedChronicle. This chronicle consists of a single table that stores the song ID (the only field in the event class schema) and time stamp of each song event. The event chronicle rule, described in the following section, inserts new rows into this table whenever new events are submitted.

Listing 6.1 shows the SongAddedChronicle declaration in the SongAdded event class.

Listing 6.1. Declaration of the Event Chronicle

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <Schema>
        ...
      </Schema>
      <Chronicles>
        <Chronicle>
          <ChronicleName>SongAddedChronicle</ChronicleName>
          <SqlSchema>
            <SqlStatement>
              --Handle update by dropping the table
              --first if it exists.
              IF EXISTS (
                  SELECT so.name
                  FROM sys.objects so
                  JOIN sys.schemas sc ON so.schema_id = sc.schema_id
                  WHERE so.name = 'SongAddedLog'
                      AND so.type = 'U'
                      AND sc.name = 'SongAlerts'
              )
                  DROP TABLE [SongAlerts].[SongAddedLog]
            </SqlStatement>
            <SqlStatement>
              -- Create the table.
              CREATE TABLE [SongAlerts].[SongAddedLog]
              (
                  SongId      INT         NOT NULL,
                  TimeAdded   DATETIME    NOT NULL
              )
            </SqlStatement>
          </SqlSchema>
        </Chronicle>
      </Chronicles>
      ...
    </EventClass>
  </EventClasses>
  ...

</Application>

Within the <EventClass> element is a <Chronicles> element that may contain one or more <Chronicle> elements. Each <Chronicle> element declares a single chronicle. For each chronicle, the <ChronicleName> element defines the chronicle name, and the <SqlSchema> element defines the SQL code that creates the chronicle tables.

Note that the chronicle declarations require you to explicitly specify the CREATE TABLE statements. This is different from the event schema declaration, for example, in which you specify a set of fields and the SQL-NS compiler generates the appropriate CREATE TABLE statement to create the events table. The reason for this difference is that, unlike the events tables, chronicle tables are completely outside the scope of the SQL-NS engine. For chronicles, you can create whatever tables you need, following any schema and structure you deem appropriate, because only your logic manipulates these tables. In contrast, the events table that the SQL-NS compiler generates from your field declarations is used by the SQL-NS engine and must have certain columns required for internal tracking and coordination.

The <SqlSchema> element can have several <SqlStatement> elements within it. Each of these statements is executed by the SQL-NS compiler in a separate batch, in the order in which they appear in the ADF. You should use separate <SqlStatement> elements whenever you need to separate SQL statements that cannot be run together in a single batch. For example, if you need to create two tables for your chronicle, each should appear in a separate <SqlStatement> element because SQL does not allow you to run two CREATE TABLE statements in a single batch.

Caution

Any statements that would need to be separated with a GO directive in tools such as Query Analyzer should be placed in separate <SqlStatement> elements. SQL-NS does not support the use of GO in any of the SQL code in the ADF.

The code in Listing 6.1 uses two <SqlStatement> elements. The first checks whether the chronicle table already exists and, if so, drops it. The second SQL statement creates the table using standard CREATE TABLE syntax. The statement that drops the table if it exists is necessary because of the way chronicle declarations are handled by the SQL-NS update tools. If you change the chronicle declaration and then update the instance, the SQL-NS compiler reruns all the code in the <SqlStatement> elements. Without the code to first drop the chronicle table if it exists, the CREATE TABLE statement could fail when executed a second time.

Caution

Although dropping the existing chronicle table allows it to be re-created on an update, think carefully about what this means for your application. Dropping the table results in any chronicled data being lost. When you update your application, if you need to preserve the data in your chronicles, you will need to back it up to another location and restore it after the update completes.

The table created to hold the chronicle data is called SongAddedLog (because it maintains a log of all SongAdded events). Data is added to the log table by the chronicle rule, as described in the next section.

Note

The schema qualified name of the chronicle table is SongAlerts.SongAddedLog because it is created in the SongAlerts schema. The schema name is specified directly in the CREATE TABLE statement that created the chronicle table. (There is no dedicated ADF element for specifying the schema name for chronicles.) In this example, the chronicle table is created in the same schema in which the SQL-NS compiler creates the other application database objects (we specified SongAlerts as the value for the <SchemaName> element in the ADF's <Database> element earlier). SQL-NS does not require you to create chronicle tables in the same schema as the other application database objects, but it's common practice to do so because chronicles are really part of the application.

Because the chronicle table has a schema qualified name, the check for its existence (in the first <SqlStatement> in the chronicle's definition) requires a join between the two system catalog views, sys.objects and sys.schemas. The sys.objects view provides a list of all the objects in the database, but does not include their schema names. The join with sys.schemas on the schema_id column obtains the schema name for each object.

Event Chronicle Rules

Event chronicle rules are SQL statements that are run when events arrive. They update the event chronicles with the appropriate data, based on the content of the events. Listing 6.2 shows the chronicle rule used to update the SongAddedLog chronicle table.

Listing 6.2. Declaration of the Event Chronicle Rule

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <Schema>
        ...
      </Schema>
      <Chronicles>
        ...
      </Chronicles>
      <ChronicleRule>
        <RuleName>UpdateSongAddedLog</RuleName>
        <Action>
          INSERT INTO [SongAlerts].[SongAddedLog](SongId, TimeAdded)
          SELECT events.SongId, GETUTCDATE()
          FROM   [SongAlerts].[SongAdded] events
        </Action>
      </ChronicleRule>
    </EventClass>
  </EventClasses>
  ...

</Application>

The <ChronicleRule> element appears after the <Chronicle> declaration. It specifies a rule name in the <RuleName> element and a SQL statement in the <Action> element. Before the SQL-NS engine runs the rule's <Action> statement, it first constructs a view over the data in the events table that contains just the events currently being processed. The view name is the same as the name of the event class (SongAdded in this case). This is the same view that the event-triggered match rule (defined in Listing 5.9 in Chapter 5, p. 155) runs against.

The chronicle rule inserts rows into the SongAddedLog table by selecting all the song IDs from the SongAdded events view. It obtains the time stamp by calling the built-in SQL GETUTCDATE() function.

Testing the Event Chronicle

Add the code shown in Listings 6.1 and 6.2 to your ADF, either by typing it in manually or using the supplementary ADF, ApplicationDefinition-5.xml, as described in the "Adding Code to the ADF" section (p. 130) in Chapter 5. When your ADF is ready, update your instance as you did before, by running the update instructions listed in the "Making Updates to an Instance and Its Applications" section (p. 131) in Chapter 5.

When the update completes, you can examine the MusicStore database and see that the SongAddedLog table is created. Now let's submit a batch of events and observe the chronicle rule executing:

1.
Open the C:\SQL-NS\Samples\MusicStore\MusicStore.ssmssln solution in Management Studio (if you don't already have it open).

2.
Open the SubmitEvents.sql script (you'll find it in the Solution Explorer, under the Queries folder in the MusicStore project).

3.
Run the SubmitEvents.sql script.

4.
Wait about 30 seconds. This is necessary because the event chronicle rule is run by the generator, which is configured to look for new event batches every 15 seconds. Depending on when in the generator's processing cycle you submit the events, it may take up to 30 seconds for the events to be processed.

5.
Open a new query window and execute the following query:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[SongAddedLog]

This returns a set of rows that show the event data just added to the chronicle table by the event chronicle rule.

Assuming that the subscriptions that you added earlier are still in your database, a set of notifications will be generated because the event-triggered subscription match rules are also run against the events you submitted (you'll see these notifications in the output file, C:\SQL-NS\Samples\MusicStore\FileNotifications.txt). Note that SQL-NS always runs an event class's event chronicle rules before it runs any event-triggered match rules. Because of this, your chronicles always are updated before the match rules are run, and you can use the chronicle data in the match rules if appropriate. (The event-triggered match rules we have in our application at this point do not use the chronicle data.)


The Scheduled Subscription Class

In this section, we implement the subscription class for the scheduled subscriptions. We begin by designing the subscription class schema and then build a subscription chronicle to keep state information about each subscription. After we have the subscription schema and chronicle defined, we can use these to implement the match rule that generates notifications.

Schema for the Subscriptions by Genre

When we designed the schema for the event-triggered subscription class (see the "Schema for the Subscriptions by Artist" section, p. 140, in Chapter 5), we began by thinking about how a user would express a subscription in words. This helped identify the subscription class fields. This technique can also be used to design the scheduled subscription class. For the subscriptions by genre, the subscription statement looks like this:

"Notify me of any new songs of genre G that have been added to the music store."

The only variable in the statement is the genre, "G." Thus, the subscription class schema needs only a single string field to hold the genre name. Listing 6.3 shows the declaration of the subscription class for the subscriptions by genre, called NewSongByGenre.

Listing 6.3. Declaration of the Scheduled Subscription Class Schema

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      ...
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>GenreName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
      ...
    </SubscriptionClass>
  </SubscriptionClasses>
  ...

</Application>

To the contents of the existing <SubscriptionClasses> element, we've added a new <SubscriptionClass> element that holds the declaration of our new subscription class. The <SubscriptionClassName> element specifies that the name of the subscription class is NewSongByGenre, and the <Schema> element defines the single field, GenreName, that stores the name of the genre in each subscription. When the subscription class is compiled, SQL-NS creates a table to store the subscription data. Following the convention we saw earlier, this table is called NSNewSongByGenreSubscriptions and contains a column for the subscription class field declared, as well as some columns used by SQL-NS internally. As you might expect, the SQL-NS compiler also creates a view over the NSNewSongByGenreSubscriptions table, called NewSongByGenre (the same name as the subscription class), to be used in subscription matching. We'll look at this view in the "Matching Logic for Scheduled Subscriptions" section, later in this chapter.

At this point, you might be wondering where the schedule information for each subscription is stored because the subscription class schema only defines a field for the genre name. Scheduled subscriptions are a built-in feature of SQL-NS, so it automatically provides all the data structures necessary for storing schedule information. SQL-NS also provides an easy way to specify a schedule when creating a subscription using the SQL-NS Subscription Management API. We'll look at this API in Chapter 7, "The SQL-NS Subscription Management API," and the internal schedule data in Chapter 11, "Debugging Notification Generation."

Subscription State

When we built the event-triggered subscription class, we had to define the subscription class schema and the notification class before we could write the match rules. For the scheduled subscription class we're now building, we will reuse the same notification class that we used before. Although the subscription data and matching criteria are different, the resulting notifications will be of the same type as those generated for the subscriptions by artist. So we have the subscription schema and notification class defined, but before we can write the match rules, we have to define the subscription state that the application has to keep. The match rules use this state information when evaluating the subscriptions against the event history.

Recall that the event chronicle implemented earlier keeps a history of all events that the application has seen. When the scheduled subscriptions are evaluated, the genres in the subscriptions are compared to the genres of the songs in the event history to determine the matches. At any given time, different subscriptions will have been evaluated against different parts of the event history. Each subscription can have a different schedule, so subscriptions can be evaluated at different times, and some are evaluated more often than others. Each time a particular subscription is evaluated, the matching logic should look only at events in the event history that arrived after the last time that subscription was evaluated. Otherwise, the application would keep regenerating notifications for old events that it had already matched.

The event chronicle keeps a time stamp for each event that indicates when that event was seen. For each subscription, we need to keep a time stamp that stores the last time it was evaluated. We can then compare the time stamp of any event in the event chronicle against the time stamp kept for the subscription to determine whether the event occurred before or after the last time the subscription was evaluated.

Just as event chronicles keep event state, SQL-NS supports subscription chronicles that hold subscription state. For our subscriptions by genre, we can create a subscription chronicle that stores a time stamp for each subscription that represents the last time the subscription was evaluated.

Event chronicles are declared as part of the event class with which they were associated. As you might expect, subscription chronicles are declared as part of the subscription class. Listing 6.4 shows the chronicle declaration in the NewSongByGenre subscription class.

Listing 6.4. Declaration of the Subscription Class Chronicle

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      ...
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <Schema>
        ...
      </Schema>
      ...
      <Chronicles>
        <Chronicle>
          <ChronicleName>NewSongByGenreChronicle</ChronicleName>
          <SqlSchema>
            <SqlStatement>
              --Handle update by dropping the table
              --first if it exists.
              IF EXISTS (
                 SELECT so.name
                 FROM sys.objects so
                 JOIN sys.schemas sc ON so.schema_id = sc.schema_id
                 WHERE so.name = 'NewSongByGenreSubscriptionProcessingTimes'
                    AND so.type = 'U'
                    AND sc.name = 'SongAlerts'
              )
                 DROP TABLE [SongAlerts].[NewSongByGenreSubscription
  ProcessingTimes]
            </SqlStatement>
            <SqlStatement>
              -- Create the table.
              CREATE TABLE [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]
          (
                  SubscriptionId     BIGINT    NOT NULL,
                  LastProcessingTime DATETIME  NOT NULL
              )
            </SqlStatement>
          </SqlSchema>
        </Chronicle>
      </Chronicles>
    </SubscriptionClass>
  </SubscriptionClasses>
  ...
</Application>

This subscription chronicle declaration has essentially the same structure as the event chronicle declaration you saw in Listing 6.1. The <ChronicleName> element provides the name of the chronicle (NewSongByGenreChronicle in this case), and the <SqlSchema> element provides the SQL statements that create the chronicle tables. The SQL-NS compiler executes each SQL statement in its own batch, in the order specified.

As in the event chronicle, the <SqlSchema> element contains two <SqlStatement> elements: the first contains a SQL statement that drops the chronicle table if it already exists; the second SQL statement creates the chronicle table. Again, this is to handle the case where the SQL-NS update tools rerun the chronicle statements against a database in which the chronicle table has previously been created. The chronicle table, NewSongByGenreSubscriptionProcessingTimes, has two columns: one for the subscription ID and one for the last processing time. For each subscription in the application, there will be a row in this table.

Unlike the event chronicle, which had a <ChronicleRules> element, there is no dedicated element in which subscription chronicle rules are defined. Instead, the subscription chronicle rules are declared along with the match rules in a single ADF element. The reason for this is that in many applications (ours included), the subscription chronicle logic is actually an inseparable part of the match logic. Parts of the chronicle update logic run before the matches are evaluated; others after. By allowing you to write the chronicle update logic along with the matching logic, SQL-NS gives you complete flexibility in the ordering and interleaving of these rules. If the chronicle rules were defined separately, it would be difficult for SQL-NS to provide sufficiently flexible execution ordering options to meet the needs of all applications. In the next section, when we look at the match rules for the subscription class, we'll also look at the code that updates the subscription chronicle.

Matching Logic for Scheduled Subscriptions

At this stage, we have all the elements necessary to write the match rules for the scheduled subscription class: the subscription schema, event state, subscription state, and notification class. Before we examine the code for the rule, it's important to understand the data available when the rule fires.

Each time this rule is run, SQL-NS populates the subscription class view, NewSongByGenre, with only those subscriptions whose schedules specify that they should be evaluated at that time. The code that determines which subscriptions should be included in the view (based on their schedules) is provided by SQL-NS. This means that match rules need not include any special logic to evaluate subscription schedules: rules can operate against the subscriptions view and they will see only the subscriptions that should be evaluated. This scoping of subscription data based on schedules is similar to the way SQL-NS scopes event data in the events view to only those events that need to be processed.

Because scheduled rules aren't associated with an event class and are not triggered by event arrival, SQL-NS does not set up an events view as it did for the event-triggered rules. The only event data available to the rule is that which it can obtain from the event chronicles.

Listing 6.5 shows the declaration of the match rules for the NewSongByGenre subscription class.

Listing 6.5. Declaration of the Subscription Class Match Rules

[View full width]
<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      ...
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <Schema>
        ...
      </Schema>
      <ScheduledRules>
        <ScheduledRule>
          <RuleName>MatchNewSongsByGenre</RuleName>
          <Action>
            -- Store the current processing time for use later.
            DECLARE @CurrentProcessingTime DATETIME
            SELECT @CurrentProcessingTime = GETUTCDATE()

            --Insert rows into Processing Times chronicle
            --for new subscriptions.
            INSERT  INTO [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]
  (SubscriptionId, LastProcessingTime)
            SELECT  subscriptions.SubscriptionId, subscriptions.Created
            FROM    [SongAlerts].[NewSongByGenre] subscriptions
            WHERE   NOT EXISTS (
                FROM    [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]chron
                SELECT  chron.SubscriptionId
                WHERE   chron.SubscriptionId = subscriptions.SubscriptionId
            )

            --Do the match.
            INSERT INTO [SongAlerts].[NewSong]
            SELECT subscriptions.SubscriberId,
                   N'DefaultDevice',
                   N'en-US',
                   songs.SongTitle,
                   songs.ArtistName,
                   songs.AlbumTitle,
                   songs.GenreName
            FROM  [SongAlerts].[SongAddedLog] eventlog
            JOIN    [Catalog].[SongDetails] songs
                ON eventlog.SongId = songs.SongId
            JOIN    [SongAlerts].[NewSongByGenre] subscriptions
                ON subscriptions.GenreName = songs.GenreName
            JOIN    [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes] chron
                ON subscriptions.SubscriptionId = chron.SubscriptionId
            WHERE eventlog.TimeAdded &gt;= chron.LastProcessingTime
                AND eventlog.TimeAdded &lt; @CurrentProcessingTime

            --Update Processing Time Chronicle for those
            --subscriptions just processed.
            UPDATE  [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]
            SET     LastProcessingTime = @CurrentProcessingTime
            FROM    [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes] chron
            JOIN    [SongAlerts].[NewSongByGenre] subscriptions
                ON chron.SubscriptionId = subscriptions.SubscriptionId
          </Action>
        </ScheduledRule>
      </ScheduledRules>
      <Chronicles>
        ...
      </Chronicles>
    </SubscriptionClass>
  </SubscriptionClasses>
  ...

</Application>

The match rules for scheduled subscriptions are declared in the <ScheduledRules> element (recall that match rules for event-triggered subscriptions are defined in the <EventRules> element). Each rule is declared in its own <ScheduledRule> element, which contains the subelements <RuleName> and <Action>. Scheduled rules do not have an <EventClassName> element as the event rules do because scheduled rules are not triggered by events. Instead, scheduled rules are executed on a periodic basis, whenever scheduled subscriptions are due to be evaluated.

The rule's logic is specified in the <Action> element. The rule logic in Listing 6.5 really does three things:

  • Inserts rows into the subscription chronicle for new subscriptions that have never been processed before

  • Matches subscriptions with the events in the event history that arrived between the last processing time and the current processing time

  • Updates the last processing times in the subscription chronicle for the subscriptions just evaluated

The first operation is handled by an INSERT statement that puts new rows into the NewSongByGenreSubscriptionProcessingTimes chronicle table. It selects those rows from the subscriptions view where the subscription ID does not exist in the chronicle table already. This is the case for new subscriptions that have never been processed before and therefore don't have a last processing time recorded. For each of these subscriptions, it inserts a row into the chronicle table that provides the subscription's creation time as the last time it was processed. This effectively initializes the chronicle record for each new subscription.

Note that the subscription ID and creation time values are obtained from the SubscriptionId and Created columns in the subscriptions view, respectively. These columns are provided by SQL-NS; for complete documentation of the columns available in the subscriptions view, consult the SQL-NS Books Online.

The next statement in the rule logic does the actual matching. It generates notifications by inserting rows into the notifications view of the NewSong notification class. The statement does a four-way join between the following:

  • The event chronicle table, SongAddedLog, which provides the event history

  • The SongDetails view in the music store catalog, which provides the full properties of the songs

  • The NewSongByGenre subscriptions view, which contains the data for the subscriptions being evaluated

  • The subscription chronicle table, NewSongByGenreSubscriptionProcessingTimes, which provides the last time each subscription was evaluated

When the subscriptions view is joined, the join condition matches the genre name specified in the subscription with the genre in the song details. Thus, only rows where the genres match are returned. This implements the fundamental matching criterion for the subscriptions by genre.

The WHERE clause filters out events that should not be evaluated at the current time. The condition specifies that the value in the TimeAdded column in the event chronicle must be greater than or equal to the last processing time recorded in the subscription chronicle and less than the current processing time. Note the use of the &gt; and &lt; escape sequences, which are required because the SQL greater than and less than operators, > and <, are reserved characters in XML.

The last statement in the rule updates the subscription class chronicle. For every subscription in the subscription class view, the statement updates the last processing time to the current processing time. This value will be used to filter out old events the next time the rule fires.

The current processing time is obtained at the beginning of the match rule (using the built-in SQL function GETUTCDATE()) and is stored in the local variable @CurrentProcessingTime. The value of this variable is used when filtering events from the event chronicle and updating the subscription chronicle's last processing times. Using a single time value like this, recorded at the beginning of the operation, avoids race conditions. Had we simply called GETUTCDATE() each time we needed the current time, the event chronicle filter and update of the subscription chronicle records may have been executed with different time values. Any events that might have arrived and been added to the event chronicle between these times would have been lost.

Notice that the scheduled rule logic does not have to examine the subscription schedules at all. Because SQL-NS determines which subscriptions are due to be evaluated and places only those subscriptions' data in the subscriptions view, the scheduled rules can just join with this view and do not need to consider the actual schedule data in any way. This makes scheduled logic easy to write and is a key benefit of using SQL-NS.

Using Event Rules and Scheduled Rules In The Same Subscription Class

SQL-NS allows you to define both event rules and scheduled rules in the same subscription class. This is useful if you want to create a single subscription class that supports both scheduled and event-triggered subscriptions.

When writing event and scheduled rules for the same subscription class, it's important to be aware that both kinds of rules execute against the same subscriptions view. The subscriptions view always contains the data from both the scheduled and the event-triggered subscriptions (regardless of which type of rule is firing).

For the event rules to execute only against the event-triggered subscriptions, and the scheduled rules only against the scheduled subscriptions, the rules must distinguish the types of subscriptions themselves. This can be done by means of the ScheduleId column that SQL-NS includes in the subscriptions view of any subscription class that has scheduled rules.

The ScheduleId column is NULL for rows that represent event-triggered subscriptions, and non-NULL for scheduled subscriptions. The event rule can include a ScheduleId IS NULL predicate in its WHERE clause to filter the scheduled subscriptions. Similarly, the scheduled rule can include a ScheduleId IS NOT NULL predicate in its WHERE clause.


Testing Scheduled Subscriptions

Add the code shown in Listings 6.3, 6.4, and 6.5 to your ADF, either by typing it in manually or using the supplementary ADF, ApplicationDefinition-6.xml, as described in the "Adding Code to the ADF" section (p. 130) in Chapter 5. When your ADF is ready, run the update instructions from Chapter 5 to update your instance.

After the update completes, you can test the scheduled subscription functionality of the application. In the prototype stage, the general technique for testing scheduled subscriptions is to enter some subscriptions that are scheduled to fire a few minutes into the future. You then submit events to ensure that the event chronicles contain an event history for the scheduled rules to operate against, and wait for the scheduled subscriptions to be evaluated. If you make refinements and need to test again, you generally enter a new set of scheduled subscriptions, again scheduled to fire a few minutes into the future. This is required because the first set of scheduled subscriptions won't fire again until the same time on the next day.

Along with the source code for this book, I've included a tool called SubscriptionLoader that can be used to add subscriptions (event-triggered or scheduled) to any SQL-NS application. We'll use this tool now to submit scheduled subscriptions to the music store application. Because SubscriptionLoader is designed to work against any SQL-NS application (it's not tied to the music store application) you'll also be able to use it when testing your own SQL-NS applications.

Note

The SubscriptionLoader tool is built using the SQL-NS Subscription Management API, which you'll learn about in Chapter 7, and the Notification Services Management Objects (NMO) API, covered in Chapter 16, "Using Notification Services Management Objects." In Chapter 16, we'll look at the source code for the SubscriptionLoader tool.

Follow these steps to add scheduled subscriptions to the music store using SubscriptionLoader and exercise the scheduled rule:

Caution

Before you perform these steps, make sure that you restarted the service, NS$MusicStore, after the last update.

1. | Invoke SubscriptionLoader.exe from the C:\SQL-NS\Tools\bin directory on your development machine.
2. | In the Connect to SQL Server dialog box, enter the name of your SQL Server and select an authentication mode. If you select SQL Server Authentication, enter the login name and password of the development account you created in Chapter 2, "Getting Set Up." Click OK to connect.
3. | The main SubscriptionLoader dialog box is divided into four sections. In the first section, you can select the SQL-NS instance, application, and subscription class for which you want to enter subscriptions. Select the MusicStore instance, the SongAlerts application, and the NewSongByGenre subscription class. (Notice that the event-triggered subscription class, NewSongByArtist, is also offered as a choice.)
4. | In the next section of the SubscriptionLoader dialog box, you can select the ID of the subscriber for whom the subscription will be created. The drop-down list should show the four subscribers we added to the music store instance in Chapter 5. Select any of these subscribers.
5. | Below the subscriber selection is a data grid in which you enter values for the subscription fields. When you select a subscription class (as you did in step 3), the grid is populated with the names of the subscription fields in that subscription class. For the NewSongByGenre subscription class, you'll see the GenreName field shown. Enter the value Jazz for the GenreName field.
6. | The final section of the SubscriptionLoader dialog box lets you specify a subscription schedule. The controls in this section are disabled if you select a subscription class with only event-triggered rules, but because NewSongByGenre contains scheduled rules, these controls should be enabled. In the Schedule Start Time field, select a start time 2 or 3 minutes into the future. Select your local time zone in the Timezone field (it should be preset to your local time zone, but you can change it as necessary). Note that the value initially shown in the Schedule Start Time field is the time at which the program was startedthis value is not updated, so you may have to add several minutes to it if you've had the program open for a while.
7. | Click the Add Subscription button. You should see a confirmation dialog box telling you that the subscription was successfully added.
8. | In Management Studio, run the SubmitEvents.sql script again to submit some new events. Make sure you do this before the start time you specified for the scheduled subscription.
9. | Wait until about 30 seconds after the start time of your scheduled subscription and then examine the notifications output file (FileNotifications.txt). You may also see this file updated before the scheduled subscription is evaluated, because the events you submitted would also have matched the event triggered subscriptions previously entered. Make sure you examine the file after the scheduled subscription's evaluation time to see the notification that was generated for it.

Although scheduled subscriptions operate against event history, and you have at least one batch of events already in your event chronicle from previous tests, it's still necessary to submit a new batch of events, as you did in step 8 of the previous instructions. This is because the first time the subscriptions are evaluated, the match rule looks only at events in the chronicle with time stamps newer than the time the subscriptions were created. (Recall that the subscription chronicle is initialized with the creation time of the subscription as the last time it was processed.) Any events in the chronicle from before the scheduled subscriptions were created are ignored.

Let's now take a look at the actual subscription data entered through the SubscriptionLoader tool and examine the contents of the chronicle tables to verify that our scheduled rule worked as expected. I intentionally avoided asking you to look at the subscriptions table immediately after adding the subscription earlier because you needed to complete the next steps before the subscription's start time was reached.

1.
From a new query window in Management Studio, execute the following query:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[NSNewSongByGenreSubscriptions]

This returns the data in the subscriptions table for all the NewSongByGenre subscriptions in the application. From this, you can see the genre names specified in each subscription. You should see a row returned for the subscription you entered in SubscriptionLoader. The subscription schedules are stored separately from the rest of the subscription data, in a table called NSNewSongByGenreSchedules. You can examine this table with the following query:

SELECT * FROM [SongAlerts].[NSNewSongByGenreSchedules]

2.
Next, you can see the events in the event chronicle and the times they were added by executing the following query:

SELECT * FROM [SongAlerts].[SongAddedLog]

This shows you all rows in the event chronicle, some of which may be older than the subscriptions.

3.
Take a look at the notification generated by querying the notifications table. Execute the following query (this returns only the notifications in the last notification batch, which was generated as a result of running the scheduled rule):

SELECT  *
FROM    [SongAlerts].[NSNewSongNotifications]
WHERE   NotificationBatchId = (SELECT MAX(NotificationBatchId)
                               FROM [SongAlerts].[NSNewSongNotifications])

Looking at the subscription data and the song details for the song IDs in the event chronicle, you should be able to see how the notification was generated.

4.
The subscription chronicle should contain a row for each scheduled subscription that shows its last processing time. Run the following query to see the contents of the subscription chronicle:

SELECT * FROM [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]

For our scheduled subscription, there should now be a row in this table that specifies the last time the subscription was processed.


Final Testing of the Prototype

We now have a complete prototype of the application: We built the event class, both subscription classes, the notification class, and the state maintenance and matching logic. We tested each piece individually as we built the application, and the last test exercised all the pieces of the application together.

Now that the prototype ADF is complete, it's important to wipe the slate clean and compile everything from scratch. When the application is deployed, it won't go through the incremental create and update process we used to develop it, so we should verify that the complete application can be created and works as expected. This ensures that no artifacts from the incremental development process affect the way the application works.

You'll find a script called cleanup.cmd in the music store sample's scripts directory that returns your system to a clean state from which you can try to compile the complete application again. This script does the following things:

  • Stops the Windows service (NS$MusicStore)

  • Deletes the instance and application

  • Unregisters the instance

  • Clears the notifications output file

  • Drops the music store database

Note that this cleanup script does not touch any of your source files, so, assuming that your ADF was complete at the time you last compiled it with the SQL-NS update tools, it should still be complete and ready to compile after running the cleanup script. The following steps describe how to run the cleanup script and recompile the whole application:

1.
From a Notification Services Command Prompt on your development machine, navigate to the scripts directory for the music store sample by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run cleanup.cmd.

3.
Run create_music_store_database.cmd. This rebuilds the music store database.

4.
Ensure that your ADF contains the complete prototype code. It should be the same as the supplementary file, ApplicationDefinition-6.xml, in the C:\SQL-NS\Chapters\06\SupplementaryFiles directory.

5.
Run create.cmd to recompile the instance and application from scratch.

6.
Run register.cmd to register the instance.

7.
Run grant_permissions.cmd to assign the appropriate database permissions.

8.
Run enable.cmd to enable the instance.

9.
Start the Windows service by typing the following command:

net start NS$MusicStore

10.
If you don't already have it open, open the music store Management Studio solution, C:\SQL-NS\Samples\MusicStore\MusicStore.ssmssln.

11.
Open the scripts AddSubscribers.sql, AddSubscriptions.sql, and SubmitEvents.sql. You'll find all three scripts in the Queries folder beneath the MusicStore project.

12.
Run AddSubscribers.sql to add subscriber and subscriber device data to the instance.

Caution

If you execute AddSubscribers.sql (or any of the other SQL scripts referenced in this section) from a query window that was previously connected to the MusicStore database before it was deleted and re-created, you may receive the following error message:

A transport-level error has occurred when sending the request to the
server. (provider: Shared Memory Provider, error: 0 - No process is on
the other end of the pipe.)

If this happens, just execute the script again and the connection will be reestablished.

13.
Run AddSubscriptions.sql to add event-triggered subscriptions to the instance.

14.
Start SubscriptionLoader (if you don't already have it open) and add a scheduled subscription of the NewSongByGenre subscription class. Set the start time to be about 2 to 3 minutes into the future.

15.
Open the notifications output file, FileNotifications.txt (from the Miscellaneous folder under the MusicStore project). Because you ran the cleanup script earlier, the notifications file should be blank.

16.
Run SubmitEvents.sql to submit some sample events. You must complete this step before the start time of the scheduled subscription you added.

17.
Within about 30 seconds, Management Studio should tell you that the notifications output file has changed. Reload the file to see the notification text. Depending on when you submitted the events, notifications for the event-triggered subscriptions and scheduled subscriptions may be generated separately. Thus, you may have to reload the notifications file twice.

When you complete all the preceding steps and get the results you expect, you can be sure that what you've built works in a basic end-to-end scenario.

Tip

In a real application, you'd typically follow this up with additional test cases that employ different samples of event and subscription data. When doing this, you'll want to pick data samples that exercise the boundary conditions in your application's logic. Remember that the purpose of the prototype stage is to ensure that the core logic of your application works, before adding the fancier input and output pieces. At the prototype stage, things are still relatively simple, so the cost of finding and fixing bugs in your application's logic will be much less than at later stages of the development process.


Summary

Scheduled subscriptions allow subscribers to specify when they should be evaluated: The evaluation of scheduled subscriptions is not triggered by event arrival. In situations where event frequency and subscription selectivity are such that event-triggered subscriptions would produce too many notifications for a subscriber to handle, scheduled subscriptions provide a convenient alternative.

To support scheduled subscriptions, applications need to keep an event history. This is because scheduled subscriptions may be evaluated long after the events have already been processed. Every time a batch of events arrives, the event history can be updated with the new event data. SQL-NS supports event chronicles as a means to maintain event history. You can create chronicle tables, as well as chronicle rules that update them, as part of the event class declaration. SQL-NS executes the chronicle rules when new events arrive.

Like event triggered subscriptions, scheduled subscriptions are represented by subscription classes. A scheduled subscription class has essentially the same structure as an event-triggered subscription class: It has a schema for the subscription data and a set of rules that do matching. However, the rules in a scheduled subscription class are declared as scheduled rules and are not associated with any event class.

In many cases, applications need to keep state about each scheduled subscription. Typically, this is used to ensure that scheduled subscriptions are evaluated only against new data added to the event history after the previous evaluation. Subscription state is maintained through subscription chronicles, and SQL-NS provides a way to declare subscription chronicles as part of a subscription class. The logic that updates the subscription chronicle tables is mixed in with the match logic in the subscription class's scheduled rules.

In this chapter, we prototyped scheduled subscriptions for the music store application. We built the event chronicles and chronicle rules required to maintain event history, and then built the scheduled subscription class. The subscription class declares a chronicle that maintains the last time each scheduled subscription was evaluated. This subscription chronicle is used in the match rule to filter out parts of the event history that have already been seen.

After testing the scheduled subscriptions, we tested the whole prototype from end to end. We deleted the instance, application, and music store databases, and cleaned up the input and output files used in testing. We then recompiled the whole instance from scratch and tested all its features. Doing this validates that the prototype is complete and works as expected. Having established this, we can continue to build on this prototype in the upcoming chapters and turn it into a real application.


Chapter 7. The SQL-NS Subscription Management API

IN THIS CHAPTER

  • Overview of Subscription Management Interfaces

  • Types of Subscription Management Interfaces

  • The SQL-NS Subscription Management API

  • Getting Set Up to Use the API

  • Implementing the Logic Layer of a Subscription Management Application

  • Building the User Interface

The primary way in which users interact with a SQL-NS application is through its Subscription Management Interface (SMI). The SMI is a custom program, developed as part of the application, that allows users to register themselves as subscribers and add, modify, and delete subscriptions. The form of the SMI varies by application: The SMI could be an ASP.NET web page, a Windows application, or a command-line program.

Whatever the nature of the interface, the SMI piece of a SQL-NS application is built using the SQL-NS Subscription Management API. This API consists of a set of classes that allow you to programmatically add, delete, and modify subscribers and subscriptions in a SQL-NS instance. When building an SMI, you focus mainly on building the user interface and business logic, and use the classes provided by SQL-NS to perform the actual data manipulations. This chapter takes a detailed look at the SQL-NS Subscription Management API and how to use it to build an SMI. Whereas the code samples in previous chapters primarily consisted of XML and T-SQL, in this chapter we'll be writing C# code in Visual Studio .NET.


Overview of Subscription Management Interfaces

In the course of using the Web, you've probably encountered several SMIs. Think about websites that offer email newsletters that you can sign up to receive or online retailers that offer to notify you when out-of-stock products become available again. Each of these presents a way for you to request that information be sent to you, either when some interesting event occurs (such as a product being restocked) or on a schedule (as in the case of weekly newsletters). This is the primary function of an SMI: to give users a way to request information in the form of notifications.

SMIs can take on many forms, but regardless of presentation, they typically provide users with an interface by which they can

  • Register as a subscriber

  • Specify how information should be to be delivered to them

  • Choose exactly what information they want and specify when it should be delivered

Registering as a subscriber establishes a record of the user in the underlying notification system. This record may contain specific information about the user, such as contact details and billing information.

When registering, subscribers usually specify the delivery mechanism for the information being requested. For example, they might choose email delivery and specify a particular email address.

Subscribers choose the information they want to receive by entering subscriptions. Depending on the application, there may be several types of subscriptions, each with various parameters that the subscriber can control. They may also specify a schedule for the delivery of the information or indicate in some other way when the information should be sent.


Types of Subscription Management Interfaces

The two main types of SMIs are

  • SMIs that allow users to create and manipulate their own subscriptions

  • SMIs that allow administrators to manage all the subscribers and subscriptions in the whole system

The same SMI can technically be used for both user and administrator functions, but generally the operations supported in each case are different enough that combining the two leads to a clumsy interface. In this chapter, we focus on SMIs for users.

The API and development techniques described in this chapter apply to both kinds of SMIs. The main difference between the two SMI types is the scope of the subscriber and subscription information they present. User SMIs generally require a user to log in and then show only the subscriber and subscription information that pertains to that user. Typically, a user cannot view or modify the subscriptions belonging to any other user. Administrator SMIs present a view of the subscribers and subscriptions in the whole system. Administrators can manipulate any user's information. Administrator SMIs may also offer reporting and monitoring capabilities over all the subscriber and subscription information. Based on the material presented in this chapter and the administration and reporting capabilities of SQL-NS presented in Chapter 14, "Administering a SQL-NS Instance," you should be able to build an administrator SMI for your application, if one is required.

Tip

In practice, almost all applications have a user SMI, but only a few have an official administrator SMI. In many cases, administrative functions that relate to subscriber and subscription information are performed with ad-hoc tools.

Architecture of Subscription Management Interfaces

Many business applications follow the three-tier model shown in Figure 7.1.

Figure 7.1. The three-tier model for business applications.

This model separates the three aspects of a business application into distinct tiers, or layers:

  • The user-interface layer implements the visual controls that users interact with.

  • The logic layer implements the business rules, policies, and transaction semantics of the application.

  • The data layer provides persistent storage and querying capabilities for the application's data.

In this model, the layers communicate with each other over well-defined interfaces. This modular approach allows layers to be changed or replaced independently of one another.

For example, if the business rules of the application changed, the user interface would not have to be rebuilt. Similarly, building an alternative user interface (perhaps for a different target audience) would not require the business logic or database schema of the application to change.

It's a good idea to follow this same three-tier model when designing an SMI for a SQL-NS notification application. Figure 7.2 shows the how the elements of an SMI map to the three tiers:

Figure 7.2. A Subscription Management Interface as a three-tier application.

[View full size image]

The user-interface layer presents visual controls by which users manipulate their subscriptions, and the data layer provides storage for the subscriber and subscription data. However, the need for the logic layer is sometimes less apparent.

To see why the logic layer is needed, let's recap what we did in the two preceding chapters. We implemented a set of subscription classes for the music store application, each of which represents a different kind of subscription. We defined the data schema for these subscription classes and the logic used to match subscriptions with events. But we didn't implement any business rules or policies around these subscription classes. Business rules and policies are outside the scope of SQL-NS and therefore are not implemented in the ADF. In a typical application, the rules and policies might specify restrictions on who can create subscriptions or the number of subscriptions that each user may have. For example, a business rule in your application might say that only "Premier Customers" can have scheduled subscriptions or that each user can have at most five subscriptions unless they pay a monthly fee. The rules and policies you implement are specific to your application. Whatever they are, they should be implemented in the logic layer of your SMI.

The data layer shown in Figure 7.2 includes the SQL-NS instance and application database objects. As we saw in Chapter 4, "Working with SQL-NS Instances," and Chapter 5, "Designing and Prototyping an Application," the subscriber information is stored in tables in the instance schema, and the subscriptions are stored in the application schema, in tables that the compiler creates for each subscription class. It's important to note that SQL-NS stores only the subscriber and subscription information it needs to perform its matching job. But, you may want to keep other information about the subscribers for different purposes. For example, your application may need to store a subscriber's billing address or privilege status. That type of information must be stored in your own database objects. The third cylinder shown in the data layer in Figure 7.2, labeled Non-SQL-NS Database Objects, represents this store of information outside the scope of the SQL-NS. In some applications, these objects may be in a separate database.

In this chapter, we build an SMI for the music store application prototyped in Chapter 5 and Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State." Our SMI follows the architectural model shown in Figure 7.2.

Much of the material in this chapter focuses on the logic layer, which communicates with the SQL-NS databases through the SQL-NS Subscription Management API. This API represents subscriber and subscription information as objects that can be programmatically manipulated. The API persists any changes to the properties of these objects by writing the appropriate data to the SQL-NS subscriber and subscription tables. The API shields you from the internal structure of these tables and the mechanics of writing to them directly. Whereas applications typically use the SQL-NS Subscription Management API to create, update, and delete subscriber and subscription information stored in the SQL-NS tables, the standard database access APIs (such as SqlClient or ADO.NET) are used to manage any other associated data in non-SQL-NS tables.

In an SMI, the logic layer should communicate with the user interface layer through a well-defined interface. The exact nature of this interface depends on the operations that the application supports. In this chapter, we define the interfaces required by the music store application and then implement the code that carries out the operations.

Note

This chapter does not go into much detail about the building of the user interface itself. We'll look at the code for a sample user interface that uses the logic layer we develop in this chapter, but there isn't much coverage of the visual parts of the user interface. The parts of the user interface code that illustrate how the user interface connects to the logic layer are discussed in more detail. The reason for this is simple: The focus of this chapter is the SQL-NS Subscription Management API. User interface development is a topic unto itself and to go into a lot of detail about that here would be a significant diversion from the SQL-NS topics that this chapter teaches.


The SQL-NS Subscription Management API

The SQL-NS Subscription Management API provides the core facilities required to build the logic layer of the SMI. This section presents an overview of the API and the classes that it contains. The next section looks at how to use this API to build the logic layer of the music store SMI.

Note

From here on, I refer to the SQL-NS Subscription Management API as the "SQL-NS API" for simplicity. Although SQL-NS does offer APIs other than those used for subscription management, and the general term SQL-NS API technically includes those other APIs as well, in the context of this chapter, we'll ignore this detail.

Fundamental Entities in the Subscription Management API

The SQL-NS API primarily focuses on three entities: subscribers, subscriber devices, and subscriptions. Before looking at the actual API, it's important that we clarify what these entities represent.

A subscriber is an entity that requests notifications by means of subscriptions. A subscriber is usually a person, although in some cases, programs can be subscribers. A subscriber is identified by a subscriber ID. All subscribers in an instance are shared among applications in that instance.

A subscriber device represents a delivery endpoint for notifications. A subscriber device might be a physical device, such as a cell phone that receives text messages, or a less tangible device, such as an email address to which notifications can be sent. Each subscriber device is associated with a subscriber, and each subscriber can have several subscriber devices. Because subscriber devices are associated with subscribers, they are stored at the instance level and can be referred to by all applications in the instance.

Each subscriber device is also associated with a delivery channel. The delivery channel represents the gateway used to get notifications to the subscriber device. As described in Chapter 4, delivery channels are configured in the ICF and shared by all applications in the instance. A delivery channel might represent, for example, a particular SMTP server used to route email notifications. In general, you have a delivery channel in your instance for each delivery system your application uses. Every subscriber device is associated with one of those delivery channels. Chapter 10, "Delivery Protocols," describes delivery channels and the ways they are used, in detail.

A subscription is a subscriber's request for a notification. Each subscription is an instance of a subscription class defined in some application and is owned by a subscriber. Unlike subscribers and subscriber devices, which are stored at the instance level, subscriptions are stored at the application level. Each subscription specifies values for the fields defined in the schema of its subscription class.

Classes in the Subscription Management API

The SQL-NS API provides classes that represent each of the entities described in the previous section. These classes allow you to examine and modify the properties of existing subscribers, subscriber devices, and subscriptions, as well as create new ones.

Table 7.1 lists the main classes in the SQL-NS API. All classes listed are defined in the Microsoft.SqlServer.NotificationServices namespace. I'll refer to this as the SQL-NS namespace from now on.

Table 7.1. Classes in the SQL-NS Subscription Management API

Class Name

Purpose

NSInstance

Represents a connection to a SQL-NS instance

NSApplication

Represents a connection to an application within a SQL-NS instance

NSException

Represents a SQL-NS exception (thrown by SQL-NS API methods in response to error conditions)

Subscriber

Represents a subscriber

SubscriberEnumeration

Provides enumeration capabilities over a collection of subscribers

SubscriberDevice

Represents a subscriber device

SubscriberDeviceEnumeration

Provides enumeration capabilities over a collection of subscriber devices

Subscription

Represents a subscription

SubscriptionEnumeration

Provides enumeration capabilities over a collection of subscriptions

TimeZone

Represents a time zone used in the specification of a schedule in a scheduled subscription

TimeZoneEnumeration

Enumerates the collection of all time zones supported by SQL-NS

Figure 7.3 shows how some of these classes are used. The NSInstance and NSApplication classes can be thought of as connectors to the SQL-NS instance and application, respectively. The other classes in the API use these connection objects to refer to a particular instance and application.

Figure 7.3. Classes in the SQL-NS Subscription Management API.

[View full size image]

The NSException class encapsulates all exceptions thrown by SQL-NS APIs. Code that uses the SQL-NS API will typically catch exceptions of the NSException type to handle SQL-NS-specific errors. Properties on the NSException class provide detailed information about the exact error that a thrown NSException object represents.

As their names suggest, the Subscriber, SubscriberDevice, and Subscription classes represent subscribers, subscriber devices, and subscriptions, respectively. Each class provides methods to create, delete, and modify instances of the entity it represents. Each class also has a corresponding enumeration class that provides strongly typed collection capabilities.

When defining the schedules for scheduled subscriptions, you'll need to specify time zones. SQL-NS provides TimeZone and TimeZoneEnumeration classes with which you can do this.

Note

This chapter shows examples of all these classes in use. However, the information in this chapter is not intended to replace or duplicate the exhaustive class reference in the SQL-NS Books Online. This chapter's examples show the use of these classes in particular contexts, but not every method and property is shown. Always consult the SQL-NS Books Online for complete information on any class you need to use. In many cases, there are properties and methods on these classes that I have not mentioned in this chapter but that may still be useful to you.


Getting Set Up to Use the API

The SQL-NS Subscription Management API is a managed code API. In other words, it is written using the .NET Framework and requires the Common Language Runtime (CLR). It can be used from any of the .NET languages (such as C#, VB .NET, or Managed C++). In this chapter, the code examples are written in C#.

Note

Although the API is written in managed code, you can still use it in native code applications through COM interop. However, this is not covered in this chapter. For more information on how to use the API through COM interop, see the SQL-NS Books Online.

This section describes the steps you need to take to get your system set up to work with the code in this chapter. The instructions here assume that you have set up your development system as described in Chapter 2, "Getting Set Up," and that you've been able to successfully build the code in previous chapters. It is also assumed that you're going to use Microsoft Visual Studio .NET to do the development work in this chapter. If you are using another development tool, you will have to adapt the Visual Studio instructions given in this chapter to your tool's environment.

The SQL-NS Assembly

The SQL-NS API is implemented in an assembly called Microsoft.SqlServer.NotificationServices.dll. In this and future chapters, I'll refer to this as the "SQL-NS assembly." When you install SQL-NS, this assembly is installed in the bin directory of your SQL-NS installation. On my system, the full path to this directory is

C:\Program Files\Microsoft SQL Server\90\NotificationServices\9.0.242\bin

On your system, the location might vary, depending on where you installed SQL-NS and the version that you installed. (At the time of this writing, version 9.0.242 is the latest version, and this is reflected in the preceding path.)

The SQL-NS assembly installed in this location is intended to be referenced at development time, to obtain the signatures of the public classes, methods, and properties in the SQL-NS API. When you create a Visual Studio project that uses the SQL-NS API, you will need to add a reference to this SQL-NS assembly. Visual Studio should be configured to locate the assembly from the appropriate path on your system automatically and show it to you in the list of available assemblies, as explained in the section that follows. If you're not using Visual Studio, you'll need to locate the assembly on your system yourself and refer to it by full path when building your code.

Referencing the SQL-NS Assembly

Any code you write that uses the SQL-NS API must reference the SQL-NS assembly. The code samples in this chapter already have the reference set up, but if you create a new Visual Studio project for any code you write on your own, you can add a reference to the assembly by performing the following steps:

1. | In Visual Studio's Solution Explorer, expand the node for your project.
2. | Right-click the References node and choose Add Reference from the context menu.
3. | In the Add Reference dialog box, select the .NET tab and scroll through the list of available assemblies to find Microsoft.SqlServer.NotificationServices.
4. | Select Microsoft.SqlServer.NotificationServices and click OK.

After doing this, you see Microsoft.SqlServer.NotificationServices listed under the References node for your project in Solution Explorer.

Tip

If the SQL-NS assembly does not appear in the list in the .NET tab of the Add Reference dialog box, you can browse for it manually by selecting the Browse tab and then navigating to the bin directory of your SQL-NS installation (described in the previous section).

Preparing the SQL-NS Instance

In this chapter, we build an SMI for the music store application that we prototyped in Chapters 5 and 6. To ensure that you're starting from the right place after the testing done in those chapters, re-create the instance from scratch.

To do this, go back to Chapter 6 and perform the instructions listed in the "Final Testing of the Prototype" section (p. 179). Pay close attention to step 5 (ensuring that your ADF contains the complete, correct prototype code). If necessary, you can always refer to C:\SQL-NS\Chapters\06\SupplementaryFiles\ApplicationDefinition-6.xml (copy this file from the CD-ROM if you changed it on your development machine), which contains the code in its correct final form. Do complete all the steps (including the testing steps, 10 through 17) just to verify that the instance is still working as expected.

The SMI Visual Studio Solution

The SMI we build in this chapter is a web front end for the music store application. Its user interface consists of a website that users can log in to and manipulate their subscriptions in the music store application. Its logic layer implements the required operations using the SQL-NS API.

The SMI is implemented as an ASP.NET 2.0 website in Visual Studio. You'll find all the source files for the SMI in the C:\SQL-NS\Samples\MusicStore\MusicStoreWeb directory. Use the following instructions to open the SMI solution (which contains the code for both the user interface and the logic layer) in Visual Studio:

1. | Start Visual Studio .NET 2005.
2. | From the File menu, choose Open, Web Site.
3. | In the panel on the left, select the File System icon and then browse the folder tree to find the C:\SQL-NS\Samples\MusicStore\MusicStoreWeb folder.
4. | Select the C:\SQL-NS\Samples\MusicStore\MusicStoreWeb folder and click Open.
5. | If the Solution Explorer is not visible, go to the View menu and select Solution Explorer (or press Ctrl+Alt+L) to open it.

Do not run the code at this time. It will not work correctly because we have not yet created some of the supporting objects required for its operation. For now you should open the solution so that you can browse through the code as you read this chapter.

Creating the Database Objects for the ASP.NET Membership Provider

As you might expect, our SMI will need to create and manage accounts for its users. ASP.NET 2.0 (on which our SMI is built), provides facilities for managing user accounts in web applications via membership providers. A membership provider is a component that stores and retrieves user account information in a particular way. In our application, we use ASP.NET's built-in SqlMembershipProvider to store user account information in the music store's SQL database. The SqlMembershipProvider implements the operations required to create, delete, and modify user accounts. Without this provider, we'd have to implement all these operations ourselves.

The SqlMembershipProvider relies on a set of database objects to store and manipulate user account information. We need to create these objects in our MusicStore database to enable it for use with the SqlMembershipProvider. Use the following steps to create the required database objects:

1.
Make sure that you've created the MusicStore instance as described in the "Preparing the SQL-NS Instance" section (p. 192).

2.
Open the script, C:\SQL-NS\Samples\MusicStore\CreateASPNETObjects.sql in Management Studio, and execute it.

The script will display several status messages as it executes. You may also see error messages with the following text:

Cannot grant, deny, or revoke permissions to sa, dbo,
 information_schema, sys, or yourself.

You can safely ignore these messages.

3.
Examine the MusicStore database in Object Explorer to see the objects created (you may need to refresh the Object Explorer tree). All the objects created by the script you just ran are in the dbo schema and have names that begin with aspnet_. For example, you'll find a table called dbo.aspnet_Users.

The objects created in this section fall into the category of non-SQL-NS objects, as depicted in the data layer in Figure 7.2. They store supporting information about subscribers that is outside the scope of the tables created by the SQL-NS compiler.

Database Permissions for Subscription Management

To perform subscription management operations, the account under which the SMI runs must have access to the database containing the instance and application database objects and be made a member of the NSSubscriberAdmin role in that database.

In the development phase, you run the SMI code under your development account (if you're using Windows Authentication, this is the Windows account under which you're logged in to your development machine; if you're using SQL Server Authentication, you'll use the SQL username and password of the development account you created in Chapter 2). Because this is also the account you used to create the MusicStore database, it is the database owner and therefore has the required permissions to connect and perform subscription management operations. It isn't necessary to explicitly grant the development account membership in the NSSubscriberAdmin role.

Note

As in previous chapters, the account and security recommendations given here apply to your development environment only. When deploying your SQL-NS application in production, you will most likely use different accounts and configure the security settings differently. Chapter 13, "Deploying a SQL-NS Instance," provides detailed information on security considerations for deployment.


Implementing the Logic Layer of a Subscription Management Application

In the "Architecture of Subscription Management Interfaces" section (p. 185), we looked at the three-tier model and why it makes sense to build SMIs this way. For the music store application's SMI, the data layer is already built. When we compiled the ICF and ADF, SQL-NS created the instance and application database objects, including tables and stored procedures for managing subscriber and subscription data. We then created the remaining objects needed in the data layer in the "Creating the Database Objects for the ASP.NET Membership Provider" section (p. 192).

This section focuses on the logic layer. It describes the purpose of the logic layer and the operations it should support, and then looks at the implementation of the music store application's logic layer using the SQL-NS API.

Purpose of the Logic Layer

The purpose of the logic layer is to implement all the operations exposed to users through the user interface. The logic layer provides a simple programming interface by which the user interface layer can invoke these operations; internally, it uses the SQL-NS API to manipulate the data layer. As such, the logic layer is an intermediary between the user interface and data layers.

Each of the SMI operations that the logic layer supports may encapsulate several operations in the SQL-NS API. You can think of each logic layer operation as a wrapper for a set of operations in the SQL-NS API that logically go together. For example, the SQL-NS API provides separate operations for adding a subscriber and adding a subscriber device. In your application, it may be the case that whenever a new subscriber is added, you add a default subscriber device. You can expose this to the user interface layer through a single logic layer operation that wraps both the adding of the subscriber and the default device.

The logic layer can also implement business rules and policies. For example, you may want to restrict each subscriber to a maximum of five subscriptions. In the logic layer operation that adds a new subscription, you could check how many subscriptions the subscriber already has and return an error if the subscriber already has the maximum number. In general, it's inconvenient to implement rules and policies of this type in the data layer. Much of the data layer is created by compiling the ICF and ADF, which don't really provide a direct way to express rules and policies.

Another function of the logic layer is to perform validation checks that would be difficult to implement elsewhere. For example, in the subscriptions by genre that our music store application supports, a user should not be able to specify a genre name in her subscription that is not one of the supported genres defined by the music store. In other words, if the music store knows only about the Jazz, Rock, and Hip Hop genres, a user should not be able to create a subscription for the Opera genre. In the logic layer operation that adds a new subscription by genre, the genre name can be validated against the list of supported genres and an error returned if necessary. Also, the logic layer could implement an operation that returned the valid list of genres; the user interface could use this to provide the user with a menu of choices he can select from when creating a subscription.

Logic Layer Operations

As mentioned earlier, the logic layer implements the operations that the user interface exposes to users. When designing the logic layer for any application, the first thing to do is list these operations. Although the exact list of operations may vary per application, in most applications, the fundamental operations are adding, modifying, and deleting subscribers, subscriber devices, and subscriptions. Typically, the subscription-related operations are implemented separately for each subscription class.

In addition to these fundamental operations, the logic layer usually provides a number of "helper operations" that serve as utilities for the user interface. These utilities typically include the following operations:

  • Check whether a subscriber already exists in the instance

  • Retrieve the set of subscribers in the instance

  • Retrieve the set of subscriptions for a particular subscriber

  • Retrieve all the possible valid values for certain subscriber and subscription propertiesfor example, the list of supported genres for the genre name property in subscriptions.

Our music store application supports all these operations. The remainder of this section illustrates the implementation of these operations in the logic layer, and in the next section, we use them in a user interface.

Logic Layer Code

We implement the logic layer of the music store SMI as a class that exposes public methods for each of the supported operations. The user interface code calls these methods to invoke the various operations. Internally, these methods call the SQL-NS API.

Note

Before we examine the actual code for the logic layer operations, it's important to note that the logic layer does not have to completely wrap the entire SQL-NS API. In other words, it's perfectly acceptable to return SQL-NS API objects from the methods on the logic layer class. In some cases, the user interface can access the properties and methods on these objects directly.

In general, you should provide wrappers only where the wrapper adds value for the user interface. In cases where the SQL-NS API already provides a class or method that the user interface can consume directly, it's better not to provide unnecessary wrapper code.

If you have not already done so, open the Visual Studio solution containing the SMI code, as described in the section "The SMI Visual Studio Solution" (p. 192). In this solution, the logic layer code is implemented in a separate source file from the rest of the user interface code. To open the logic layer source file, expand the App_Code folder in the Solution Explorer and double-click the MusicStoreSMIOperations.cs file.

The logic layer class is called MusicStoreSMIOperations. Glance through the MusicStoreSMIOperations.cs source file to see the methods and properties it implements. In the remainder of this section, we'll look at each of these in detail. I'm not going to show the complete source code listing for this class in one place in the text. Instead, I'm going to highlight pieces of it as I explain each concept. As I did in Chapter 6, I'll use abbreviated code listings that show pieces of the class code in context, but I'll omit code that doesn't relate to the immediate discussion. In each abbreviated listing, omitted code is indicated with an ellipsis (...). This same technique is used in other code examples in subsequent chapters of this book.

Connecting to the Data Layer: The Instance and Application Objects

The operations that the logic layer implements involve manipulating data in the instance and application database objects. The SQL-NS API provides the NSInstance and NSApplication classes to facilitate connecting to these database objects. These classes allow you to connect to the databases containing the objects, just by specifying the instance and application names. You do not need to know where those databases are located or what they're called; NSInstance and NSApplication read the Registry keys installed when the instance is registered (with the nscontrol register command line tool or the Register Instance command in Management Studio) to obtain the information required to establish a database connection.

The logic layer declares NSInstance and NSApplication objects as private members and instantiates them in its constructor. These are then used by the other operations. Listing 7.1 shows the declaration of the NSInstance and NSApplication objects and the code for the class constructor:

Listing 7.1. Constructor That Creates the NSInstance and NSApplication Objects

public class MusicStoreSMIOperations
{
    // Constant strings used throughout this class.
    private const string INSTANCE_NAME = "MusicStore";
    private const string APPLICATION_NAME = "SongAlerts";
    ...

    // Private member variables.
    private NSInstance musicStoreInstance;
    private NSApplication songAlertsApp;

    /// <summary>
    /// Just initializes the connection to the instance and application.
    /// </summary>
    public MusicStoreSMIOperations()
    {
        try
        {
            musicStoreInstance = new NSInstance(INSTANCE_NAME);

            songAlertsApp = new NSApplication(
                musicStoreInstance,
                APPLICATION_NAME);
        }
        catch (NSException ex)
        {
            string errorMsg = String.Format(
                "Unable to connect to {0} instance and {1} application: {2}",
                INSTANCE_NAME,
                APPLICATION_NAME,
                ex.Message);

            throw new Exception(errorMsg, ex);

         }
      }
      ...
}

The instance and application names are defined in constant strings declared at the top of the class. Then, an NSInstance object called musicStoreInstance and an NSApplication object called songAlertsApp are declared.

The MusicStoreSMIOperations constructor instantiates the NSInstance class with the new operator, passing the instance name to the NSInstance constructor. If you configured your development environment for SQL Server Authentication when setting up the source code in Chapter 2, this line of code on your system will differ slightly from what's shown here in Listing 7.1. Specifically, the version of the code on your system will pass two additional arguments (not shown in the listing here) to the NSInstance constructor: the SQL username and password to use when connecting to the database. The values will be hard-coded to the username and password for the SQL development account you created in Chapter 2. For example, if the username is "SQL-NS_Dev" and the password is "devPassword," the line that creates the musicStoreInstance object would read as follows:

musicStoreInstance = new NSInstance(INSTANCE_NAME, "SQL-NS_Dev", "devPassword");

The API uses the given SQL credentials to connect to the database for all subsequent operations. If you're using Windows Authentication, no username or password is passed to the NSInstance constructor, so the code reads exactly as shown in Listing 7.1. With Windows Authentication, the API connects to the database using the credentials of the Windows account under which the SMI runs.

Caution

From a security perspective, it's not a good idea to hard-code usernames and passwords in your code. This technique is used here (in the case of SQL Server Authentication) only for simplicity of illustration.

If you're using SQL Server Authentication in a production application, you should design a scheme by which your application can be configured with the appropriate SQL username and password without compromising security. A common technique is to encrypt the username and password and store them in a Registry key. Your code can read this Registry key, decrypt the values, and then set the SqlUser and SqlPassword properties.

When implementing any scheme that encrypts values, it's recommended that you use the Windows Data Protection API (DP API). You can find more information on this API in the MSDN documentation or online at http://msdn.microsoft.com/library/enus/dnsecure/html/windataprotection-dpapi.asp.

After the musicStoreInstance object is created, the constructor creates a new NSApplication object, songAlertsApp, with the new operator. It passes the instance object and the application name as parameters.

The creation of the NSInstance and NSApplication objects can fail for a number of reasons. For example, the instance may not be registered, the SQL Server may be unreachable, or the username and password supplied for SQL authentication may be incorrect. If an error does occur, an NSException is thrown. In the constructor code, the creation of the instance and application objects happens with a TRy block that catches exceptions of type NSException. The catch block simply constructs a meaningful error message that the user interface can display and then throws a new exception with this error message and the original NSException as the inner exception.

Manipulating Subscribers

The MusicStoreSMIOperations class implements a method called AddSubscriber() to add a new subscriber record to the system. When a new user wants to register as a subscriber, the user interface invokes this method. Listing 7.2 shows the code for AddSubscriber().

Listing 7.2. Implementation of the AddSubscriber() Method

public class MusicStoreSMIOperations
{
...
    public Subscriber AddSubscriber(string subscriberId, bool enabled)
    {
        Subscriber s = new Subscriber(musicStoreInstance);

        // Set the properties on the subscriber object.
        s.SubscriberId = subscriberId;
        s.Enabled = enabled;

        // Write the subscriber record to the database.
        s.Add();

        // Also add a default device for the subscriber.
        AddSubscriberDevice(s);

        return s;
     }
...
}

AddSubscriber() takes two parameters: a subscriber ID string and a Boolean flag indicating whether the subscriber record should be marked as enabled when it is created. The subscriber ID string becomes the subscriber ID in the new subscriber record.

The first line of code creates a new instance of the Subscriber class, which is provided by the SQL-NS API to represent subscribers. When creating the new Subscriber object, it passes the instance object (created in the constructor as shown in Listing 7.1) to the Subscriber constructor. This is the pattern followed by almost all the objects in the SQL-NS API: The constructor takes either an NSInstance or an NSApplication object, depending on whether the object represents an entity that is part of the SQL-NS instance or a particular application. Recall that subscribers are stored at the instance level, so the Subscriber constructor takes an instance object.

The next two lines set the SubscriberId and Enabled properties on the subscriber object. The SubscriberId property is a string and should specify a unique identifier for the subscriber in the instance. The format of the subscriber ID is entirely up to you, as long as each subscriber has a unique ID. Typically, applications use an identifier for the subscriber ID that serves as a key into other records related to the subscriber. For example, if your system already keeps a profile for each user, keyed by user ID, that user ID can be used as a subscriber ID. As you'll see later when we look at the user interface code, our application uses the username managed by the ASP.NET SqlMembershipProvider (our user account management system) as the subscriber ID.

The Enabled property behaves like an on/off switch for the subscriber. If a subscriber is enabled, SQL-NS processes its subscriptions and sends it notifications. If a subscriber is disabled, SQL-NS does not do any processing for it: The subscriber's subscriptions are not evaluated when events arrive, and any undelivered notifications are not sent. As you see in this chapter, several SQL-NS entities, including subscribers, subscriber devices, and subscriptions, can be enabled or disabled.

When the subscriber properties (SubscriberId and Enabled) are set, the code then calls Add() on the subscriber object. This causes the subscriber record to be written to the database. Until the Add() method is called, the subscriber record and its properties exist only in memory. The final line of code in AddSubscriber() calls AddSubscriberDevice(), another operation method on our logic layer class, which adds a default subscriber device for the subscriber. We look at this operation in the "Manipulating Subscriber Devices" section (p. 204). Note that by calling AddSubscriberDevice() here, our logic layer ensures that every subscriber added gets a default subscriber device. The adding of a subscriber and adding of a subscriber device are combined into a single operation from the perspective of the user interface layer.

After the subscriber device is added, the AddSubscriber() operation is complete. The method returns the newly created subscriber object to the caller.

Subscriber Records and Subscriber Details

For each subscriber, SQL-NS maintains a record that contains the subscriber ID and enabled state. All other entities associated with the subscriber (its subscriptions and subscriber devices) are related via the subscriber ID. SQL-NS does not need to keep any other information about subscribers, but other parts of the application may well need to. For example, it may be necessary to store a customer profile and billing information for each subscriber.

If your application needs data about a subscriber beyond what SQL-NS stores, you must store this data in non-SQL-NS database objects. In general, you define the schema for the detail data and create the corresponding database structures (tables, views, and stored procedures) separately from your SQL-NS development. In the case of using the ASP.NET SqlMembershipProvider, the database objects required to store details about users can be generated automatically: the script you ran in the "Creating the Database Objects for the ASP.NET Membership Provider" section was generated by running a tool called aspnet_regsql that is shipped with ASP.NET. For more information on this tool, refer to the ASP.NET 2.0 documentation.

In some cases, the subscriber detail information is written to the non-SQL-NS objects as part of the process of adding a subscriber. In our application, this is not the case: user accounts are created separately from adding subscribers. However, in other applications, it's not uncommon for an operation such as AddSubscriber() to implement both the creation of the subscriber record using the SQL-NS API and the writing of the additional subscriber details using other data access APIs.

Modifying Subscriber Records

The only property of a subscriber record that can be changed after it is created is the Enabled property. You cannot change the SubscriberId, because this is the unique identifier for the subscriber.

The logic layer class does not provide an explicit operation to support modifying the Enabled property of the subscriber. This is because the SQL-NS API already provides an interface for performing this operation: You simply change the value of the Enabled property on the subscriber object and call the Update() method on it. Because the subscriber object is returned from the AddSubscriber() method (and can be obtained later via a subscription enumeration), the user interface can perform the update on this object directly. As mentioned earlier, it's not necessary to completely wrap the SQL-NS API in the logic layer code; where it makes sense to do so, you can allow the user interface to manipulate SQL-NS API objects directly. Changing a subscriber's enabled property is an example of such a case.

Removing Subscriber Records

SMI user interfaces commonly provide users with a way to remove their subscriber registrations. To support this, the logic layer code has to provide an operation to remove subscriber records.

The MusicStoreSMIOperations class provides a RemoveSubscriber() method for this purpose. As you'll see from the code, this method is another example of the logic layer code encapsulating several separate SQL-NS API operations into a single logic operation that the user interface can invoke. Listing 7.3 shows the code for RemoveSubscriber().

Listing 7.3. Implementation of the RemoveSubscriber() Operation

public class MusicStoreSMIOperations
{
...
    public void RemoveSubscriber(Subscriber s)
    {
        bool failedBecauseDisabled = false;

        try
        {
           // Attempt to delete the subscriber.
           s.Delete();
        }
        catch (NSException ex)
        {
            // If the delete failed because the subscriber was disabled,
            // we're going to enable it and then try again, so we'll set
            // a flag that we'll check later. If it failed for any other
            // reason, we'll just throw the exception out to the caller.
            if (ex.ErrorCode == NSEventEnum.SubscriberDisabled)
            {
                failedBecauseDisabled = true;
            }
            else
            {
                throw;
            }
        }

        // If the delete failed because the subscriber was disabled,
        // enable it and then try the delete again.
        if (failedBecauseDisabled)
        {
            // We do the enable in its own try block, so that we isolate
            // any exceptions that occur from this operation and form a
            // meaningful error message to the caller.
            try
            {
                // Set the enabled bit and update the subscriber.
                s.Enabled = true;
                s.Update();
             }
             catch (NSException ex)
             {
                 throw new Exception(
                     "Failed to enable subscriber before deleting: " +
                     ex.Message,
                     ex);
             }

             // Now try the delete again. If this fails, the exception
             // will just get thrown out to the caller.
             s.Delete();
        }
    }
...
}

RemoveSubscriber() is passed a subscriber object that represents the subscriber record to be removed. It's reasonable to expect callers (the user interface code) to pass in a subscriber object because subscriber objects are returned from the AddSubscriber() method, and the logic layer provides a way to obtain a subscriber object for an existing subscriber, as we'll see later.

SQL-NS semantics dictate that a subscriber must be enabled before it is deleted. This makes sense from the perspective of consistency, but it's not necessarily something you'd want to expose in the user interface. Therefore, the logic layer operation, RemoveSubscriber(), encapsulates the code that enables a subscriber before deleting it, hiding this perhaps counterintuitive SQL-NS requirement from the user interface. RemoveSubscriber() provides the user interface an easy facility for removing a subscriber, regardless of its enabled state.

In the first try block in the code shown in Listing 7.3, the Delete() method on the subscriber object is called. This attempts to delete the subscription. If this fails, an NSException is thrown. If the subscriber record failed to be deleted because the subscriber was disabled, the NSException has an error code that indicates this.

The catch block checks the error code in the NSException that was caught. If the code indicates that the subscriber was disabled, it sets a flag indicating that this was the cause of failure. Later, the code checks this flag to see whether another attempt should be made. If the error code in the exception object indicates any other problem, the exception is simply rethrown to the caller (causing the whole operation to fail).

Note

All NSException objects contain an error code that you can access via the ErrorCode property. All the error codes are defined in the enumeration NSEventEnum. The values in this enumeration are documented in the SQL-NS Books Online, but not in much detail because many of them are internal error codes used only within the SQL-NS engine. However, in certain cases where the error code is useful to calling applications, the SQL-NS Books Online points out values that you should look for.

If the delete operation failed because the subscriber was enabled, the next try block attempts to enable the subscriber. It first sets the Enabled property to true and then calls the Update() method to write this changed value to the database. These operations appear in their own TRy block, because if they fail, we want to construct a meaningful error message for the caller. Simply allowing any thrown exceptions to bubble up to the caller would be confusing; the caller might not be expecting an error having to do with enabling a subscriber, because the enable operation is hidden inside the implementation of the remove operation. Instead, the code catches the exception, and in the catch block, constructs an error message that says that it failed to enable the subscriber before attempting to delete it. The original NSException is passed as the inner exception to the new Exception object that is constructed and thrown.

If the enabling of the subscriber succeeds, we try to call the Delete() method on the subscriber again. If this fails, we allow the resulting exception to be thrown to the caller.

You might wonder why it's necessary to attempt the delete to determine whether the subscriber is disabled, rather than simply checking the Enabled property on the subscriber object passed in. The reason is that the object may contain stale data. All objects in the SQL-NS API are in-memory caches of data in the instance or application databases. If the underlying data in the database changes, the snapshot reflected in the objects may not reflect reality. Thus, we cannot trust the value of the Enabled property: The enabled state of the underlying subscriber record may have been changed by the user running another instance of the SMI, or by an administrator.

Another approach would have been to always enable the subscriber before attempting the delete. However, this would have required two round-trips to the database for every remove operation. In cases where the subscriber was already enabled (the common case), only one round-trip is really necessary. The way the code is currently written, we make additional round-trips to the database only where absolutely necessary. In the common case, the subscriber is enabled, and the first delete succeeds.

Manipulating Subscriber Devices

Each subscriber can have several devices to which notifications can be delivered. Each device is represented by a subscriber device record in the instance database, which specifies the device type, address, and delivery channel.

Your SMI probably needs to provide a way for subscribers to add, modify, and delete their subscriber device records. In the logic layer, you can implement any business rules associated with subscriber devices. For example, you could stipulate that every subscriber must have one default subscriber device and ensure that this subscriber device record is created whenever a new subscriber is added (the code in our music store example does this). You could also, for example, restrict the user of certain delivery channels that offer premium service to those subscribers who have paid a fee for such service.

In addition to providing an implementation of the subscriber device business rules, the logic layer can also provide the decision logic that chooses an appropriate delivery channel based on choices the user made in the user interface. For example, suppose that your application supports sending notifications via email and via Short Message Service (SMS), the text messaging protocol used by cell phones and pagers. You would probably have two delivery channels, each of which represented a connection to a delivery system that supported one of the two methods of delivery. The names given to these delivery channels may not be meaningful to the user; all the user knows is whether he wants his notifications sent via email or via SMS. The logic layer could expose a method that took the delivery method as a parameter and chose the appropriate delivery channel name based on this.

At this point, our music store instance has only one delivery channel: the one configured to use the File delivery protocol. Using additional delivery channels is not explained until Chapter 10, so at this point it's difficult to show much interesting code around subscriber devices and delivery channel choices. For now, I'll show the logic layer code that adds the default subscriber device. When we look at delivery channels in more detail in Chapter 10, I'll describe the corresponding SMI logic layer code to support them.

Adding Subscriber Devices

As you saw in Listing 7.2, the AddSubscriber() operation adds a default subscriber device by calling AddSubscriberDevice(), another method on the MusicStoreSMIOperations class. Listing 7.4 shows the implementation of this method.

Listing 7.4. Implementation of the AddSubscriberDevice() Method

public class MusicStoreSMIOperations
{
...
    public SubscriberDevice AddSubscriberDevice(Subscriber s)
    {
        SubscriberDevice device = new SubscriberDevice(musicStoreInstance);

        // Set the device properties. We use the default values for the
        // single delivery channel in the application.
        device.SubscriberId = s.SubscriberId;
        device.DeviceName = "DefaultDevice";
        device.DeviceTypeName = "File";
        device.DeviceAddress = "";
        device.DeliveryChannelName = "FileChannel";

        // Add the device.
        device.Add();

        return device;
     }
...
}

AddSubscriberDevice() takes the subscriber object representing the subscriber for whom the default device should be added. It instantiates a new SubscriberDevice object, passing the instance object as a parameter to the SubscriberDevice constructor. It then sets several properties on the subscriber device object: the subscriber ID, device name, device type, device address, and delivery channel name.

Don't be concerned if these properties are unfamiliar at this time. For the moment, note that the subscriber device is associated with a particular subscriber (by subscriber ID) and delivery channel (by delivery channel name).

The subscriber device itself has a name, which can be any string by which you can meaningfully identify the device when displaying it in the user interface. The device name must be unique within the scope of the owning subscriber. In other words, a single subscriber cannot have two devices with the same name, but two different subscribers could each have a device with the same name.

Note

We'll look at the other subscriber device properties in more detail in later chapters. We'll cover device types when we look at content formatting in Chapter 9, "Content Formatters," and device addresses when we look at delivery protocols in Chapter 10.

After the subscriber device properties have been set, the code calls the Add() method to write the subscriber device record to the database. AddSubscriberDevice() then returns the newly created subscriber device object to the caller.

Modifying and Removing Subscriber Devices

The subscriber device object provides a Delete() method that can be used to remove a subscriber device record from the instance database. Also, it provides an Update() method that writes any changes made to the object's properties back to the subscriber device record in the database. Because at this stage our application has a single delivery channel and every subscriber must have a default device configured to use this channel, it does not make sense to remove or modify subscriber devices. Therefore, our logic layer and user interface do not implement these operations; you will not see the subscriber device Delete() or Update() methods being called.

Manipulating Subscriptions

The most important function of an SMI is to provide users with a way to manipulate their subscriptions. The ADF defines the subscription classes that represent the kinds of subscriptions the application supports, and the SMI code described in this section creates, deletes, and modifies subscriptions of these subscription classes.

The SQL-NS API provides a Subscription class that you can use to perform operations on subscription data. This is used in the SMI logic class, to implement operations that manipulate subscriptions. In the implementation of these operations, you can encode business rules such as restrictions on the number of subscriptions that subscribers can have, or validation of the subscription parameters.

Note

The terms "Subscription class" and "subscription class" can sometimes be confused. Subscription is a class in the SQL-NS API that represents individual subscriptions. A subscription class in a SQL-NS application represents a particular kind of subscription supported by the application. The Subscription class can be used to manipulate subscriptions of any subscription class.

Listing 7.5 shows the implementation of the AddNewSongByArtistSubscription() method of the MusicStoreSMIOperations class. This method adds a new subscription of the NewSongByArtist subscription class, defined in Chapter 5. The user interface calls this method when it needs to add a new subscription of this subscription class.

Listing 7.5. Implementation of the AddNewSongByArtistSubscription() Method

public class MusicStoreSMIOperations
{
...
    private const string      NEW_SONG_BY_ARTIST_SUBSCRIPTION_CLASS_NAME = "NewSongByArtist";
...
    public Subscription AddNewSongByArtistSubscription(
        Subscriber  s,
        string      artistName)
    {
        // Create a new subscription object for a subscription of
        // the NewSongByArtist subscription class.
        Subscription subscription = new Subscription(
            songAlertsApp,
            NEW_SONG_BY_ARTIST_SUBSCRIPTION_CLASS_NAME);

        // Set the subscriber id.
        subscription.SubscriberId = s.SubscriberId;

        // Set the fields in the subscription class
        // schema (we have only one).
        subscription["ArtistName"] = artistName;

        // Add the subcription;
        subscription.Add();

        return subscription;
    }
...
}

AddNewSongByArtistSubscription() takes a subscriber object and an artist name as arguments. The subscriber object represents the subscriber for whom the subscription should be added, and the artist name provides a value for the subscription class field, ArtistName, declared in the ADF. Had the subscription class schema defined more fields, this method would have needed additional arguments to obtain values for them.

The first statement in the method creates a new instance of the Subscription class, passing the application object and the subscription class name as arguments to the constructor. The subscription class name comes from a constant defined as a private member of the MusicStoreSMIOperations class.

After creating the subscription object, the subscriber ID property is set using the value from the subscriber object. The next line supplies a value for the ArtistName field. Note that this field is not represented as a property on the object in the same way as the subscriber ID. Remember that Subscription is a generic class that is supposed to represent subscriptions of any subscription class. Because each subscription class may have a different set of fields (as declared in the ADF), the generic Subscription class cannot possibly define properties to represent them all.

Instead, field values are set by means of the index operator ([]) on the subscription object, passing the field name as the index. In the case of the code in Listing 7.5, subscription["ArtistName"] refers to the ArtistName field in the subscription. The value provided for this field is obtained from the argument passed to the AddNewSongByArtistSubscription() method. After setting the properties and fields on the subscription object, the Add() method is called to write the subscription data to the subscription table in the application database. Finally, the newly created subscription object is returned to the caller.

Tip

One advantage of implementing operations in your logic layer to add subscriptions, instead of having the user interface call the SQL-NS API directly, is that you can ensure that the values for the subscription fields are set correctly. Using the SQL-NS API directly, there is no guarantee that all the required fields will be supplied or that the values supplied will be of the correct type. By implementing an add method that takes the required set of fields as strongly typed parameters and sets them appropriately in the subscription object, you greatly reduce the opportunity for error.

Specifying Subscriptions Schedules

The code in Listing 7.5 adds an event-triggered subscription. For event-triggered subscriptions, the only values that need to be set on the subscription object are the subscriber ID and the subscription class fields. However, when adding scheduled subscriptions, you also need to provide a subscription schedule. Listing 7.6 shows the code for the operation that adds a subscription of the NewSongByGenre scheduled subscription class.

Listing 7.6. Implementation of the AddNewSongByGenreSubscription() Method

public class MusicStoreSMIOperations
{
...
    private const string NEW_SONG_BY_GENRE_SUBSCRIPTION_CLASS_NAME = "NewSongByGenre";
...
    public Subscription AddNewSongByGenreSubscription(
        Subscriber  s,
        string      genreName,
        DateTime    scheduleStartTime,
        Microsoft.SqlServer.NotificationServices.TimeZone timeZone)
    {
        // Create a new subscription object for a subscription of the
        // NewSongByGenre subscription class.
        Subscription subscription = new Subscription(
            songAlertsApp,
            NEW_SONG_BY_GENRE_SUBSCRIPTION_CLASS_NAME);
            // Set the subscriber id.
            subscription.SubscriberId = s.SubscriberId;

            // Set the fields in the subscription class
            // schema (we have only one).
            subscription["GenreName"] = genreName;

            // Set the subscription schedule.
            subscription.ScheduleRecurrence = "FREQ=DAILY;";
            subscription.ScheduleStart = FormatScheduleStart(
                scheduleStartTime,
                timeZone);

            // Add the subcription;
            subscription.Add();

            return subscription;
      }
...
}

In addition to the subscriber object and the genre name (which is used as the value for the GenreName subscription class field), AddNewSongByGenreSubscription() also takes a schedule start time and a time zone object. These two arguments are required to construct the subscription schedule.

As the name suggests, the schedule start time specifies the first time that the subscription should fire. The time zone object is an instance of the TimeZone class in the SQL-NS API that represents the time zone in which the schedule start time is expressed. Note that we have to use the fully qualified class name (including the namespacespecifier) for the TimeZone class. This is because there is also a .NET Framework class called TimeZone in the System namespace. The use of the Microsoft.SqlServer.NotificationServices namespace disambiguates the class name.

The AddNewSongByGenreSubscription() method creates a new subscription object, passing the application object and the subscription class name to the constructor. Note that the same SQL-NS API class, Subscription, is used to represent both event-triggered and scheduled subscriptions. After instantiating the subscription object, the code sets the subscriber ID property and the value of the subscription field, GenreName.

Thereafter, two properties that relate to the subscription schedule, ScheduleRecurrence and ScheduleStart, are set. The ScheduleRecurrence property specifies how often the subscription should fire, after the time of the first firing, which is specified in ScheduleStart.

The value of the ScheduleRecurrence property is a string that indicates the recurrence pattern for the schedule. The simplest and most frequent recurrence pattern is daily recurrence. When this pattern is specified, the subscription fires at the same time every day. Other recurrence patterns are

  • Weekly On particular days of the week

  • Monthly On a particular day of the month, in particular months of the year

  • Yearly On a particular day of a particular month

The string value assigned to ScheduleRecurrence follows the ICalendar recurrence rule syntax (ICalendar is an Internet standard for calendaring and scheduling). In the code shown in Listing 7.6, we assign the value, "FREQ=DAILY;", which is the ICalendar syntax for daily recurrence. The syntax for specifying the other recurrence patterns in the preceding list is given in the SQL-NS Books Online (see the documentation for the ScheduleRecurrence property of the Subscription class).

The value of the ScheduleStart property is just a formatted string version of the start time and time zone passed in. The format is specified in detail in the SQL-NS Books Online in the documentation for the ScheduleStart property of the Subscription class. The code in Listing 7.6 calls a helper method, FormatScheduleStart(), to properly format the schedule start string. Listing 7.7 shows this function.

Listing 7.7. Implementation of the FormatScheduleStart() Helper Method

public class MusicStoreSMIOperations
{
...

    private string FormatScheduleStart(
        DateTime scheduleStartTime,
        Microsoft.SqlServer.NotificationServices.TimeZone   timeZone)
    {
        return String.Format("TZID={0}:{1}T{2}",
            timeZone.TimeZoneId.ToString().Trim(),
            scheduleStartTime.ToString("yyyyMMdd"),
            scheduleStartTime.ToString("HHmmss"));
    }
...
}

This method takes the start time and time zone as parameters and returns a string formatted according to the following notation:

TZID=<TimeZoneId>:yyyyMMddTHHmmss

In this notation, <TimeZoneId> is the identifier for the time zone, obtained from the TimeZoneId property on the time zone object. yyyy, MM, and dd are the parts of the date, and HH, mm, and ss are the parts of the time. The date and time portions of the string are separated by the "T" character. To format the date and time correctly, this method uses the ToString() method on the .NET Framework DateTime structure, which takes a format specifier string as an argument.

Caution

Although the specification of the schedule start time includes both date and time portions, the date portion is ignored by the SQL-NS engine. Just the time value is used to determine the time of day at which the subscription should be evaluated on the days specified by the schedule recurrence.

Going back to the code in Listing 7.6, you see that after the schedule recurrence and start time are set, the Add() method on the subscription object is called to write the subscription record, including the schedule, to the database. The method then returns the newly created subscription object. Notice that the code for adding a scheduled subscription is similar to the code for adding an event-triggered subscription. The only difference is that for scheduled subscriptions you need to set the ScheduleRecurrence and ScheduleStart properties on the subscription object.

Modifying and Removing Subscriptions

Like the other objects in the SQL-NS API, the subscription object provides Delete() and Update() methods that can be used to remove and modify the underlying subscription records. Because we are returning subscription objects from the logic layer operations that add subscriptions, we do not need to implement explicit logic layer operations to delete and modify subscriptions. The user interface layer can call the subscription object's Delete() and Update() methods directly. You would implement wrappers for these methods in the logic layer only if you needed to implement business logic associated with these operations.

Helper Operations in the Logic Layer

In addition to the fundamental operations that we just examined, the logic layer also provides a number of helper methods that serve as convenient utilities for the user interface layer. In this section, we look at the helper methods implemented on the MusicStoreSMIOperations class.

Retrieving All the Subscribers in an Instance

The SQL-NS API provides a SubscriberEnumeration class that is used to represent a collection of subscriber objects. The enumeration class provides methods to iterate over the individual subscriber objects in the collection and access a particular object via the index operator.

The logic layer should provide an operation that creates a SubscriberEnumeration containing all the subscribers in the instance and returns this to the caller. The user interface can use this operation whenever it needs to obtain a collection of all subscribers, such as when it needs to display them in a list.

In the MusicStoreSMIOperations class, this operation is implemented as a property, as shown in Listing 7.8.

Listing 7.8. Implementation of the Subscribers Property

public class MusicStoreSMIOperations
{
...

     public SubscriberEnumeration Subscribers
     {
         get
         {
             return new SubscriberEnumeration(musicStoreInstance);
         }
     }
...
}

The get accessor for the property creates and returns a new SubscriberEnumeration object, passing the instance object to the constructor. This method of instantiating the SubscriberEnumeration class results in a collection that contains all the subscribers in the instance.

Note

The data in the SubscriptionEnumeration class is populated lazily. In other words, the data isn't actually fetched from the database until the individual items in the collection are enumerated. Because of this design, creating a SubscriberEnumeration is not an expensive operation, even if an instance has thousands (or even millions) of subscribers.

Checking Whether a Subscriber Exists

A common operation in SMI user interfaces is checking whether a subscriber record exists for a given subscriber ID. This can be used, for example, to determine whether a given user is already an existing subscriber or has to register as a subscriber for the first time.

The MusicStoreSMIOperations class implements a Boolean function called SubscriberExists() that takes a subscriber ID parameter and returns true only if a subscriber record for that subscriber ID exists. Listing 7.9 shows the code for this function.

Listing 7.9. Implementation of the SubscriberExists() Function

public class MusicStoreSMIOperations
{
...

     public bool SubscriberExists(string subscriberId)
     {
         bool result = false;

         try
         {
             // Try to access the subscriber record using the subscriber id
             // as an index.
             Subscriber s = this.Subscribers[subscriberId];
             result = true;
         }
         catch (IndexOutOfRangeException)
         {
             // Catching an IndexOutOfRange exception when indexing the
             // subscriber enumeration means the record doesn't exist.
             result = false;
         }

         return result;
     }
...
}

The first line in the try block attempts to obtain a subscriber object for the given subscriber ID. Let's dissect this line of code:

            Subscriber s = this.Subscribers[subscriberId];

By referencing this.Subscribers, this line is actually invoking the get accessor of the Subscribers property we looked at in Listing 7.8. This returns a SubscriberEnumeration containing all the subscribers in the instance. The subscriber enumeration is then being indexed via the subscriber ID. SubscriberEnumeration provides an index operator that takes a subscriber ID as a string and returns the corresponding subscriber object.

If there is no subscriber object in the collection with the given subscriber ID, the index operator throws an IndexOutOfRangeException. The code in Listing 7.9 catches this exception and uses it as an indication that the subscriber record does not exist. In the catch block, it sets the local result variable to false.

If no exception is thrown by the index operator, it means that a subscriber object with the given subscriber ID was found. Control proceeds to the next line, which sets the result variable to true. At the end of the function, the value of the result variable is returned.

Retrieving Subscriptions

The user interface often needs to display the existing subscriptions that a given subscriber has. To do this, it needs to obtain a collection of the subscription objects that represent those subscriptions. It can then read the properties of these subscription objects to display the subscription data.

The MusicStoreSMIOperations class implements two functions that return subscription collections, each of which supports one of the two subscription classes in the application. Listing 7.10 shows these functions.

Listing 7.10. Implementation of the Methods That Return the Subscriptions for a Given Subscriber

public class MusicStoreSMIOperations
{
...

    public SubscriptionEnumeration GetNewSongByArtistSubscriptions(
        Subscriber s)
    {
        SubscriptionEnumeration subscriptions = s.GetSubscriptions(
            songAlertsApp,
            NEW_SONG_BY_ARTIST_SUBSCRIPTION_CLASS_NAME);

        return subscriptions;
     }
     ...
     public SubscriptionEnumeration GetNewSongByGenreSubscriptions(
         Subscriber s)
     {
         SubscriptionEnumeration subscriptions = s.GetSubscriptions(
             songAlertsApp,
             NEW_SONG_BY_GENRE_SUBSCRIPTION_CLASS_NAME);

         return subscriptions;
     }
...
}

Both functions take a subscriber object as input and return a SubscriptionEnumeration object. The SubscriptionEnumeration class provides the same enumeration capabilities for Subscription objects that the SubscriberEnumeration class does for Subscriber objects.

Let's examine the GetNewSongByArtistSubscriptions() function first. As its name suggests, this function returns an enumeration of the subscription objects representing the given subscriber's NewSongByArtist subscriptions.

The Subscriber object provides a GetSubscriptions() method that retrieves subscriptions. GetSubscriptions() takes an application object and a subscription class name as parameters and returns an enumeration of the subscriber's subscriptions in that subscription class.

GetNewSongByArtistSubscriptions() calls the GetSubscriptions() method on the subscriber object it is passed. It supplies the application object and the name of the NewSongByArtist subscription class as parameters. The enumeration object obtained is then returned to the caller.

The GetNewSongByGenreSubscriptions() function is almost identical. The only difference is the subscription class name passed to the GetSubscriptions method on the subscriber object.

Implementing helper methods like these raises the level of abstraction at which the user interface code can operate. It does not need to know the subscription class names or manage the NSApplication objects required to retrieve the data.

Data Collections

There are cases where the SMI needs to display collections of data from which the user can make a selection. The logic layer provides these data collections for the user interface's use.

In the music store application, there are two such collections of data that the user interface needs to display. One is the list of genres supported by the music store, and the other is the list of time zones that can be used in a subscription schedule.

The MusicStoreSMIOperations class provides properties that return collections representing the set of genres and the set of time zones. Listing 7.11 shows the code for these properties.

Listing 7.11. Implementation of Data Properties Used by the User Interface

public class MusicStoreSMIOperations
{
...

    public TimeZoneEnumeration TimeZones
    {
        get
        {

            return new TimeZoneEnumeration(musicStoreInstance, "en");
        }
    }
    ...

    public string[] Genres
    {
        get
        {
            // NOTE: We could actually obtain the values by reading them
            //       from the music store database.
            return new string[3]{"Jazz", "Rock", "Hip Hop"};
        }
    }
...
}

The TimeZones property returns a TimeZoneEnumeration object. TimeZoneEnumeration is a class in the SQL-NS API that provides enumeration capabilities over objects of its TimeZone class. The get accessor for the TimeZones property implemented in Listing 7.11 creates and returns a new TimeZoneEnumeration object. The TimeZoneEnumeration constructor takes the instance object and a language name as parameters. It returns an enumeration of TimeZone objects whose property values are expressed in the given language. In this simple example, we pass in the identifier "en" to represent English because that's the only language the simple user interface in this chapter will support.

The Genres property returns an array of strings that represent the genre names currently supported by the music store. For the sake of simplicity, I've just hard-coded the genre names. They could also have been read from the actual music store database. This would definitely be the more robust solution because it would allow support for new genres to be added without the logic layer or user interface having to change. But because I'm trying to focus on the SQL-NS APIs here and not digress into general database access code, I've just shown the simplified, hard-coded implementation.


Building the User Interface

This section presents a sample user interface built on the logic layer developed in the previous section. The coverage here focuses on how the user interface employs the logic layer and the SQL-NS API itself to implement subscription management. It does not cover user interface programming fundamentals.

Note

Although only parts of it are shown in this chapter, the complete source code for the user interface is available in the SMI Visual Studio solution, described in the section "The SMI Visual Studio Solution" (p. 192). This code is thoroughly commented, so reading through it should give you a good idea of how the user interface works.

User Interface Technologies

A number of user interface frameworks are available for you to use when building an SMI. The most common ones are ASP.NET and Windows Forms. ASP.NET allows you to build an SMI as a web application, and Windows Forms allows you to build an SMI as a Windows application.

Both ASP.NET and Windows Forms work with managed code. You can also build an SMI user interface using native code technologies such as ASP (the predecessor to ASP.NET) and Microsoft Foundation Classes (MFC). To do so, you have to use the SQL-NS API through COM interop, as described in the SQL-NS Books Online.

The user interface we look at in this section is a website, built with ASP.NET 2.0, that presents a simple front end to the music store. The site does let you browse the music store's catalog, but most of its functionality is designed to facilitate managing subscriptions in the music store's notification application. For simplicity, other aspects of a typical music store, such as ordering music or playing sample songs, are not implemented in our website.

We'll examine the code behind the website's user interface for managing subscriptions and test it using the local web server built into Visual Studio 2005. You will not need a full Internet Information Server (IIS) installation to run the website.

Note

Although deep knowledge of ASP.NET development is not required to understand the material in this section, some familiarity with the basic concepts is assumed. To learn more about ASP.NET and web development, I recommend the tutorials in the MSDN library and those available online.

Running the Sample User Interface

Before we analyze the code, you should run it and play with the user interface to get a feel for how it works. This section describes the various parts of the interface and walks you through using it to create and manipulate subscriptions. To begin, open the SMI solution in Visual Studio (if you haven't already done so), go to the Debug menu and select Start Debugging (or press F5) to build and run the SMI website.

The website starts on the home page, which displays the list of songs in the music store catalog. This list is read from the database when the page is loaded. Only 10 songs are displayed at a time; to see more songs, use the page navigation links at the bottom of the song list.

A list of links is displayed in the left pane. When you first start the website, you'll see two links in the list: one labeled Login, which leads to a login page for existing users, and another labeled Register New User, leading to a page on which new users can create an account.

Click the Register New User link to create the first account in the music store website. The registration page presents a form in which you specify the new account details. Enter a username, password, email address, and security question and answer for the new account. The username can be any name you choose; it does not need to correspond to an existing Windows or SQL username. When you're done filling out the form, click the Create User button to create the account.

Tip

If clicking the Create User button resulted in an ASP.NET error page with a message that reads

   Could not find stored procedure 'dbo.aspnet_CheckSchemaVersion'.

or another similar error message, you probably have not created the database objects required by the ASP.NET SqlMembershipProvider. Go back and complete the instructions given in the section "Creating the Database Objects for the ASP.NET Membership Provider" (p. 192) and then try to create the user again.

Caution

The scripts we used to test the music store application previously created subscribers called "Bob", "Mary", "Alex", and "Emily". Do not use any of these as usernames when creating accounts through the SMI website. If you do, you'll encounter an error later on when the SMI attempts to create a subscriber record for the account (because records with these names already exist).

After the account is created, the website will display a status message telling you that the account was created successfully. Click the Continue button to proceed. This redirects you back to the home page. The user account just created is now logged in. Notice that the list of links has changed: the Login link has been replaced with a Logout link and a new link labeled Manage Subscriptions has replaced the Register New User link. As you can see, the list of links changes based on whether a user is logged in. If you log out, the links will revert back to the ones you saw earlier, before you logged in.

Click the Manage Subscriptions link to open the page with the main subscription management user interface. Figure 7.4 shows a screenshot of this page.

Figure 7.4. The user interface for managing subscriptions.

[View full size image]

The page is divided into two main sections: one for managing subscriptions by artist and the other for managing subscriptions by genre. Each section has a set of controls for adding new subscriptions, a list of existing subscriptions, and a set of buttons to enable, disable, and remove those existing subscriptions.

To see the page in action, enter a NewSongByArtist subscription by typing an artist name into the text box labeled Artist Name and then clicking the Add Subscription button. The artist name you entered will be added to the list of current subscriptions. You can enable, disable, or remove this subscription by selecting it in the list and clicking the appropriate button on the right.

Add a NewSongByGenre subscription by selecting a genre from the Genre Name drop-down list, specifying a schedule time and time zone using the other drop-down lists, and then clicking the Add Subscription button. The genre name will be added to the list of existing subscriptions by genre. Like the subscriptions by artist, subscriptions by genre can also be enabled, disabled, or removed by selecting the genre name in the list and clicking one of the buttons on the right.

You can see the results of the operations you performed through the interface by querying the subscription tables in the database. Run the following queries in Management Studio to see all the NewSongByArtist and NewSongByGenre subscriptions:

USE MusicStore
SELECT * FROM [SongAlerts].[NSNewSongByArtistSubscriptions]
SELECT * FROM [SongAlerts].[NSNewSongByGenreSubscriptions]

Notice the Enabled column in the results of these queries that shows the enabled state for each subscription. When you enable or disable subscriptions through the interface and then rerun these queries, the value in the Enabled column will change. Before continuing, spend some time adding and manipulating additional subscriptions through the user interface and observe that the changes you expect are made to the records in the database.

Examining the User Interface Code

The SMI website is implemented in a series of ASP.NET web forms. Each form is defined in a .aspx file and supported by C# code in a corresponding .aspx.cs file. You can see these files in the SMI solution using the Visual Studio Solution Explorer. The controls for subscription management operations are laid out in the ManageSubscriptions.aspx web form (this is the page that loaded when you clicked the Manage Subscriptions link in the website). The operations behind these controls are implemented in C# code in the corresponding ManageSubscriptions.aspx.cs file. In this section, we focus on the parts of this code that use the logic layer methods we implemented earlier and those that use the SQL-NS API directly. Other parts of the website code are not covered here because they do not relate specifically to SQL-NS.

Open the ManageSubscriptions.aspx.cs file in Visual Studio (you may need to expand the ManageSubscriptions.aspx node in the Solution Explorer tree to see the ManageSubscriptions.aspx.cs file). The code in this file implements parts of the ManageSubscriptions class, which provides event handlers and other supporting code for the controls in the ManageSubscriptions.aspx page. In the following sections, we'll look at specific methods in the ManageSubscriptions class.

Initializing the Page

The ManageSubscriptions class implements an event handler called Page_Load(). As its name suggests, this handler is called whenever the ManageSubscriptions.aspx page is loaded. It performs the initialization of all the components in the page. Listing 7.12 shows the code in the Page_Load() handler.

Listing 7.12. Implementation of the Page_Load() Event Handler

public partial class ManageSubscriptions : System.Web.UI.Page
{
    private MusicStoreSMI.MusicStoreSMIOperations smiOperations;

    protected void Page_Load(object sender, EventArgs e)
    {
        try
        {
            // Connect to the SMI logic layer.
            smiOperations = new MusicStoreSMI.MusicStoreSMIOperations();

            if (!IsPostBack)
            {
                // Check if a subscriber record exists for the currently
                // logged in user. If not, create one. The subscriber id
                // is the user name.

                string subscriberId = User.Identity.Name;

                if (!string.IsNullOrEmpty(subscriberId))
                {
                    if (!smiOperations.SubscriberExists(subscriberId))
                    {

                        Subscriber subscriber = smiOperations.AddSubscriber(
                            subscriberId, true);
                    }
                    else
                    {
                        // Display this subscriber's subscriptions.
                        PopulateSubscriptions(subscriberId);
                    }
                }
                else
                {

                    // The user name was null or empty, meaning
                    // there is no logged in user. Redirect to the
                    // login page.
                    Response.Redirect("~/Login.aspx");
                }

                // Populate the list of genres shown in the dropdown.
                PopulateGenres();

                // Populate the drop-down lists used to specify schedules.
                PopulateScheduleControls();
             }
         }
         catch (Exception ex)
         {
             HandleInternalError(ex);
         }

    }
    ...
}

As Listing 7.12 shows, The ManageSubscriptions class declares an instance of the logic layer class MusicStoreSMIOperations as a private member:

    private MusicStoreSMI.MusicStoreSMIOperations smiOperations;

The Page_Load() handler creates a new instance of this class and assigns it to the private member:

       smiOperations = new MusicStoreSMI.MusicStoreSMIOperations();

This instance of the logic layer class is used to invoke the logic layer operations and helper methods throughout the user interface code.

Recall that in its constructor, the logic layer class MusicStoreSMIOperations established connections to the instance and application databases and threw an exception if these connections could not be created. The instantiation of the logic layer class in the Page_Load() handler appears within a TRy-catch block that catches any exceptions and displays an appropriate error message. As you'll see in subsequent sections, other methods in the ManageSubscriptions class also use a similar TRy-catch block to catch and handle exceptions generated by other logic layer operations.

After instantiating the logic layer class, the Page_Load() handler performs a series of steps that initialize the controls on the page (if the page load is not a post back). The first step is to obtain the name of the currently logged in user. This is used as the subscriber ID for SQL-NS purposes. ASP.NET makes the username of the currently logged in user available through the User.Identity.Name property. If this property returns a null or empty string, no user is currently logged in. In this case, the page redirects to the login page of the website.

If there is a currently logged in user, the next step calls the SubscriberExists() helper function on the logic layer class to determine whether a subscriber record already exists for the user. If no subscriber record exists, the code invokes the AddSubscriber() operation on the logic layer class to create a new subscriber record. As you saw in Listing 7.2, this operation also adds a subscriber device for the subscriber.

If a subscriber record already exists for the currently logged in user, Page_Load() calls a helper method, PopulateSubscriptions(), to display the user's subscriptions in the two list boxes you saw earlier. We'll look at the code for PopulateSubscriptions() in the next section.

The remaining code in the Page_Load() handler calls helper methods to populate the other controls on the page (the drop-down lists used to select genres and specify subscription schedules). I won't go into the details of these helper methods here, but you should look at the code in the source file. Notice that the PopulateGenres() helper method uses the enumeration of genres we implemented on the logic layer class.

Displaying Subscriptions

As you saw in the previous section, when the ManageSubscriptions.aspx page is loaded, its Page_Load() handler calls a helper method, PopulateSubscriptions(), to display the existing subscriptions of the currently logged in user. Listing 7.13 shows the implementation of PopulateSubscriptions().

Listing 7.13. Implementation of the PopulateSubscriptions() Helper Method

public partial class ManageSubscriptions : System.Web.UI.Page
{
    ...
    private void PopulateSubscriptions(string subscriberId)
    {
        // Obtain the subscriber object.
        Subscriber subscriber = smiOperations.Subscribers[subscriberId];

        // Get an enumeration of the NewSongByArtist subscriptions.
        SubscriptionEnumeration subscriptions =
            smiOperations.GetNewSongByArtistSubscriptions(subscriber);

        // Clear any items currently in the list box.
        ListBoxNewSongByArtistSubscriptions.Items.Clear();

        // Enumerate the subscriptions and add the artist names to
        // the list box.
        foreach (Subscription s in subscriptions)
        {
            ListBoxNewSongByArtistSubscriptions.Items.Add(
                (string)s["ArtistName"]);
        }

        // Get an enumeration of the NewSongByGenre subscriptions.
        subscriptions =
            smiOperations.GetNewSongByGenreSubscriptions(subscriber);

        // Clear any items currently in the list box.
        ListBoxNewSongByGenreSubscriptions.Items.Clear();

        // Enumerate the subscriptions and add the genre names to the
        // list box.
        foreach (Subscription s in subscriptions)
        {
            ListBoxNewSongByGenreSubscriptions.Items.Add(
                (string)s["GenreName"]);
        }
    }
    ...
}

PopulateSubscriptions() takes a subscriber ID as a parameter and displays, in the page's two list boxes, all the NewSongByArtist and NewSongByGenre subscriptions belonging to that subscriber. The code begins by obtaining the Subscriber object for the given subscriber ID from the Subscribers enumeration on the logic layer object.

The logic layer method GetNewSongByArtistSubscriptions() is called to obtain an enumeration of the subscriber's NewSongByArtist subscriptions. The code then iterates over the Subscription objects in this enumeration, adding the value of the ArtistName field in each one to the first list box. In a similar way, the GetNewSongByGenreSubscriptions() method on the logic layer object is called to obtain the NewSongByGenre subscriptions, and the values of the GenreName fields are displayed in the other list box.

Adding Subscriptions

When a user clicks one of the Add Subscription buttons in the user interface, control is routed to the click event handler for the button. Listing 7.14 shows the implementation of the click event handler for the Add Subscription button in the subscriptions by artist area of the user interface.

Listing 7.14. Implementation of the Click Event Handler for the Button That Adds Subscriptions by Artist

public partial class ManageSubscriptions : System.Web.UI.Page
{
    ...
    protected void ButtonAddNewSongByArtistSubscription_Click(
        object sender,
        EventArgs e)
    {
        try
        {
             // Obtain the artist name from the text box.
             string artistName = TextBoxArtistName.Text.Trim();

             if (!string.IsNullOrEmpty(artistName))
             {
                 // Obtain the subscriber id from the user name. We
                 // can safely use this here because we're guaranteed
                 // to have a logged in user (the page load handler
                 // redirects anonymous access to the login page).
                 string subscriberId = User.Identity.Name;

                 // Obtain the subscriber object from the logic layer.
                 Subscriber subscriber =
                     smiOperations.Subscribers[subscriberId];

                 // Add a subscription for this subscriber.
                 smiOperations.AddNewSongByArtistSubscription(
                     subscriber, artistName);

                 // Add the artist name to the list box showing
                 // current subscriptions.
                 ListBoxNewSongByArtistSubscriptions.Items.Add(artistName);

                 // Change the status display.
                 ShowSuccessStatusInLabel(
                 LabelAddNewSongByArtistSubscriptionStatus,
                 "Subscription added successfully.");
             }
             else
             {
                 ShowErrorStatusInLabel(
                     LabelAddNewSongByArtistSubscriptionStatus,
                     "You must specify an artist name.");
             }
        }
        catch (Exception ex)
        {
            HandleInternalError(ex);
        }
    }
    ...
}

The event handler first obtains the value entered in the Artist Name text box. If this value is null or an empty string, an error message is displayed telling the user that an artist name must be specified.

If the artist name read from the text box is a non-empty string, the code proceeds to add the subscription. It obtains the subscriber ID of the currently logged on user (again, via the User.Identity.Name property) and then gets the corresponding Subscriber object from the enumeration returned by the logic layer's Subscribers property. The next step calls the AddNewSongByArtist method on the logic layer object, passing the subscriber object and the artist name. Finally, the code updates the list of current subscriptions in the list box by adding the new artist name and displays a status message indicating success.

The click event handler for the button that adds NewSongByGenre subscriptions is very similar to the one shown in Listing 7.14. The main difference is that NewSongByGenre subscriptions require a schedule to be specified, so the code must read the start time and time zone from the appropriate controls in the user interface and pass these into the AddNewSongByGenreSubscription() method on the logic layer. You should look at the code for the handler, called ButtonAddNewSongByGenreSubscription_Click(), in the source file.

Enabling, Disabling, and Removing Subscriptions

In the subscription management page, the buttons to the right of the list boxes allow users to enable, disable, or remove subscriptions. As you might expect, the click event handlers for these buttons implement these operations. Listing 7.15 shows the implementation of the click event handler for the button that enables NewSongByArtist subscriptions.

Listing 7.15. Implementation of the Click Event Handler for the Button That Enables Subscriptions by Artist

[View full width]
public partial class ManageSubscriptions : System.Web.UI.Page
{
    ...
    protected void ButtonEnableNewSongByArtistSubscription_Click(
        object sender,
        EventArgs e)
    {
        try
        {
            Subscription subscription =
                GetSelectedNewSongByArtistSubscription();

            if (null != subscription)
            {
                // Set the enabled property.
                subscription.Enabled = true;

                // Update the subscription.
                subscription.Update();

                // Change the status display.
                ShowSuccessStatusInLabel(
                    LabelRemoveNewSongByArtistSubscriptionStatus,
                    "Subscription enabled successfully.");
            }
            else
            {
                ShowErrorStatusInLabel(
                LabelRemoveNewSongByArtistSubscriptionStatus,
                "You must select an artist name from the list to enable the associated
   subscription.");
            }
        }
        catch (Exception ex)
        {
            HandleInternalError(ex);
        }
    }
    ...
}

First, the click event handler obtains the Subscription object for the subscription currently selected by calling a helper method, GetSelectedNewSongByArtistSubscription(). Before we continue, let's take a look at the implementation of this method, shown in Listing 7.16.

Listing 7.16. Implementation of the GetSelectedNewSongByArtistSubscription() Helper Method

public partial class ManageSubscriptions : System.Web.UI.Page
{
    ...
    private Subscription GetSelectedNewSongByArtistSubscription()
    {
        Subscription subscription = null;

        // Obtain the artist name from the text box.
        string artistName =
            ListBoxNewSongByArtistSubscriptions.SelectedValue;

        if (!string.IsNullOrEmpty(artistName))
        {
            // Obtain the subscriber id from the user name. We
            // can safely use this here because we're guaranteed
            // to have a logged in user (the page load handler
            // redirects anonymous access to the login page).
            string subscriberId = User.Identity.Name;

            // Obtain the subscriber object from the logic layer.
            Subscriber subscriber = smiOperations.Subscribers[subscriberId];

            // Find the subscription object.
            SubscriptionEnumeration subscriptions =
                smiOperations.GetNewSongByArtistSubscriptions(subscriber);

            foreach (Subscription s in subscriptions)
            {
                if ((string)s["ArtistName"] == artistName)
                {
                    subscription = s;

                    break;
                }
            }
        }
1
        return subscription;
    }
    ...
}

GetSelectedNewSongByArtistSubscription() begins by obtaining the selected artist name from the list box. If no artist name was selected (indicated by a null or empty string being returned from the list box's SelectedValue property), the method returns null.

If an artist name was selected, the code proceeds to find the associated Subscription object. It does this by enumerating the subscription objects for the currently logged on user's NewSongByArtist subscriptions and comparing the ArtistName field in each one to the selected artist name. When a match is found, the associated Subscription object is returned.

Going back to Listing 7.15, after the call to GetSelectedNewSongByArtistSubscription(), the click event handler checks the returned value. If it's null (meaning no subscription was selected), it displays an error message telling the user to select a subscription. Otherwise, it sets the Enabled property on the subscription object to true and calls the Update() method.

The code for disabling subscriptions is almost the same. The click event handler for the button that disables NewSongByArtist subscriptions, ButtonDisableNewSongByArtistSubscription_Click(), differs from Listing 7.15 only in the line that sets the Enabled property on the subscription object; it sets Enabled to false, instead of true.

Note

The Enabled property on a subscription object behaves like an on/off switch for the subscription. When a subscription is disabled, it is never evaluated when events arrive. However, undelivered notifications generated when the subscription was enabled may still be delivered after the subscription becomes disabled.

The code shown in Listing 7.15 uses the Update() method on the subscription object to write a change to the Enabled property back to the underlying subscription record in the application database. Update() can also be used to write back changes to other properties, including the values of the subscription fields. This is useful if your SMI supports editing existing subscriptions.

Notice that enabling and disabling subscriptions are cases in which the user interface invokes the SQL-NS API directly to perform subscription management operations. There are no wrappers in the logic layer class for enabling or disabling subscriptions, because the API provides this functionality in a way that's convenient to use directly.

Another example of the user interface calling the SQL-NS API directly is in the code that removes a subscription. Listing 7.17 shows the click event handler for the button that removes NewSongByArtist subscriptions:

Listing 7.17. Implementation of the Click Event Handler for the Button That Removes Subscriptions by Artist

[View full width]
public partial class ManageSubscriptions : System.Web.UI.Page
{
    ...
    protected void ButtonRemoveNewSongByArtistSubscription_Click(
        object sender,
        EventArgs e)
    {
        try
        {
            Subscription subscription =
                GetSelectedNewSongByArtistSubscription();

            if (null != subscription)
            {
                // Delete the subscription.
                subscription.Delete();

                // Remove the artist name from the list box.
                ListBoxNewSongByArtistSubscriptions.Items.Remove(
                    (string)subscription["ArtistName"]);

                // Change the status display.
                ShowSuccessStatusInLabel(
                    LabelRemoveNewSongByArtistSubscriptionStatus,
                    "Subscription removed successfully.");
            }
            else
            {
                ShowErrorStatusInLabel(
                LabelRemoveNewSongByArtistSubscriptionStatus,
                "You must select an artist name from the list to remove the associated
   subscription.");
            }
        }
        catch (Exception ex)
        {
            HandleInternalError(ex);
        }
    }
    ...
}

The structure of this event handler is similar to the one that enables subscriptions, shown in Listing 7.15. It uses the same code pattern to obtain the Subscription object for the currently selected subscription. It then calls the Delete() method on the subscription object, which removes the associated subscription record from the database. Finally, it updates the list box of current subscriptions by removing the artist name in the deleted subscription from the list and displaying a status message to the user.

The click event handlers for the buttons that enable, disable, and remove NewSongByGenre subscriptions follow the same patterns as the ones for NewSongByArtist subscriptions shown in this section. Take a look at the code in ManageSubscriptions.aspx.cs to see how these handlers are implemented.


Summary

The Subscription Management Interface of a SQL-NS application serves as the front end that its users interact with. A well-architectured SMI consists of three distinct layers: a user interface layer that presents visual controls to the user, a logic layer that implements the operations supported by the SMI in accordance with the appropriate business rules and policies, and a data layer in which subscriber and subscription data is persisted.

SQL-NS provides a Subscription Management API that can be used to manipulate subscriber and subscription data in the instance and application database objects. This API consists of a set of classes that represents subscribers, subscriber devices, and subscriptions.

The SMI data layer is created when the ICF and ADF files are compiled to form the instance and application database objects. When building the SMI for an application, you create the logic and user interface layers, as well as any application-specific objects in the data layer required to store subscription information not managed by SQL-NS.

In the logic layer, you implement methods for the operations that the SMI needs to support, as well as helper methods that the user interface layer can use. The implementation of the logic layer uses the classes in the SQL-NS API.

The user interface layer implements the look and feel of the SMI. It provides visual controls that the user can interact with and internally invokes the facilities provided by the logic layer. In some cases, it also invokes the SQL-NS API directly.

In this chapter, we built the logic layer of the SMI for the music store application and looked at a sample user interface implementation. We'll use the SMI built here to enter subscriptions in the music store application as we proceed with its development in the upcoming chapters.


Chapter 8. Event Providers

IN THIS CHAPTER

  • Overview of Event Providers and Event Submission

  • Event Provider Concepts

  • Preparing to Work with the Source Code in This Chapter

  • Using the Built-in Event Providers

  • APIs for Building Custom Event Providers

  • Building Custom Hosted Event Providers

  • Building Standalone Event Providers

  • Cleanup: Preparing for the Next Chapter

This chapter explores event providers, the components that submit events into SQL-NS applications for processing. In the sample applications you've seen so far, you've submitted events manually, via TSQL scripts. In effect, you have been the event provider for these applications. Manually submitting events this way is a useful technique during testing because it allows you to hand pick the event data the applications process. But eventually, most applications need one or more real event providers to gather events from event sources automatically.

SQL-NS comes with a set of built-in event providers: configurable components that can be reused in any application. It also provides a framework for building custom event providers. This chapter describes the built-in event providers in detail and also teaches techniques for building various types of custom event providers. All the examples in this chapter extend the music store application we've been developing since Chapter 5, "Designing and Prototyping an Application."


Overview of Event Providers and Event Submission

In the context of SQL-NS applications, an event is usually some occurrence that is potentially interesting to a group of subscribers. For a SQL-NS application to match events against subscriptions, it needs to be given data that describes the events. Event providers are the components that detect events and submit the corresponding data about those events to SQL-NS applications. You can think of event providers as connectors between SQL-NS applications and event sources. A SQL-NS application may have several event providers submitting events from various event sources.

Note

SQL-NS terminology refers to the input of event data as event submission. Rarely is the distinction between the event data and the event itself called out; the term event is usually used to mean event data.

SQL-NS supports several kinds of event providers. In any SQL-NS application, the application domain and the sources of events usually determine the form of the event providers used. This section provides an overview of the various types of event providers and how to use them in SQL-NS applications. The remainder of this chapter goes into more detail about these event provider types.

Event Providers as Components Hosted in the SQL-NS Engine

As you've seen in previous chapters, the engine that drives a SQL-NS instance usually runs in a Windows service and connects to the database to perform processing. The SQL-NS engine hosts the running components of the instance's applications, including event providers, generators, and distributors.

Event providers that run inside the SQL-NS engine are called hosted event providers and can be thought of as part of the engine itself. SQL-NS provides three built-in hosted event providers: the FileSystemWatcherProvider, the SQLProvider, and the AnalysisServicesProvider. Each of these is described in more detail later in this chapter.

The framework for running event providers in the SQL-NS engine is extensible, so you can build your own custom hosted event providers. You write the code that implements the event provider operations, and the SQL-NS engine invokes it. Custom hosted event provider code is written in the form of a class that implements a SQL-NS event provider interface. In the ADF, you configure the SQL-NS application to use the event provider class. When the SQL-NS engine starts, it loads the event provider class and calls the event provider interface methods.

SQL-NS supports two kinds of custom hosted event providers: continuous and scheduled. Continuous event providers are started when the SQL-NS engine starts and run until the engine stops (or until an error condition forces them to stop). Scheduled event providers are started when the SQL-NS engine starts and then are periodically invoked according to a schedule configured in the ADF. The type of event provider appropriate for a given application depends on the event sources and the application semantics. The "Choosing an Event Provider Type" section (p. 265) later in this chapter provides some guidance on when to choose one type over the other.

Event Providers as Standalone Applications

SQL-NS also supports event providers that run as standalone applications, outside the SQL-NS engine. When implemented this way, event providers can take on any form and need not implement any particular interface mandated by SQL-NS.

Note

Standalone event providers are sometimes referred to as nonhosted event providers because they are not hosted within the SQL-NS engine. I find the term "nonhosted" a little awkward and prefer to use standalone when describing these event providers.

SQL-NS does not provide any built-in standalone event providers. If you want to use a standalone event provider in your application, you have to build it yourself. Building an event provider as a standalone application, rather than as a component hosted with the SQL-NS engine, offers several dimensions of flexibility. First, the event provider is not constrained within any particular framework or process model. Standalone event providers also give you complete control over their lifetime because they are not started and stopped with the SQL-NS engine. This usually makes them easier to invoke manually, for debugging or testing purposes. Furthermore, if you choose the standalone approach, the event provider functionality can be embedded within a larger application. In other words, although a standalone event provider can be a dedicated program whose sole purpose is to submit events, it can also be a feature in an application that has another primary purpose.

Standalone event providers can be written as console programs, Windows applications, web applications, or database applications. SQL-NS provides event submission APIs that can be used in all these kinds of applications. The "APIs for Building Custom Event Providers" section (p. 260) later in this chapter provides an overview of all the event submission APIs, and the code examples throughout the remainder of this chapter show them in use.

Building and Configuring Event Providers

If you're building a custom event provider, you first write the code that implements its operations and then configure it in the ADF. If you're using a built-in provider, you don't have to write any code, but you must still configure the provider in the ADF to customize it for your particular application.

When building a custom event provider, you then have a choice of three APIs that you can use to implement its event submission operations:

  • The Event Object API, which represents each event as an object

  • The XML Bulkload API, which reads event data from XML files

  • The SQL-NS event submission stored procedures, which allow you to submit events from SQL code

These three APIs are described in more detail in the "APIs for Building Custom Event Providers" section (p. 260), later in this chapter.

Configuration information for event providers is specified in the <Providers> element of the ADF. Two types of subelements can go within the <Providers> element: <HostedProvider> and <NonHostedProvider>. The <HostedProvider> element declares a hosted event provider (one that runs within the SQL-NS engine), and the <NonHostedProvider> element declares a standalone event provider. You've already seen an example of the <NonHostedProvider> element in the configuration of the sample applications in earlier chapters.

For a hosted event provider, the configuration specifies the information that the SQL-NS engine needs to load the event provider on startup. For standalone event providers, the configuration is minimal: Only the event provider name is specified. We'll look at both the <HostedProvider> and <NonHostedProvider> elements in more detail as we work with the various event providers in this chapter.

You can add event providers to an existing application by adding the appropriate <HostedProvider> or <NonHostedProvider> elements to the ADF and then updating the instance (with nscontrol update or the Update Instance command in Management Studio). We'll use this technique in this chapter as we develop new event providers.


Event Provider Concepts

This section describes some of the key concepts related to using and building event providers. These concepts form the basis for the material covered in the rest of this chapter.

Event Providers and Event Sources

SQL-NS applications typically deal with two kinds of event sources:

  • Active event sources that push events out to interested parties

  • Passive event sources from which interested parties pull events

For example, a traffic alerting system that broadcasts incident reports to listeners is an active event source. In contrast, a web service that clients have to poll to obtain stock market data is a passive event source.

Note

Active event sources may actually be implemented as notification applications. The output (notifications) of one notification application may well be the input (events) of another.

Both types of event sources interface with SQL-NS applications via event providers, but event providers behave differently depending on whether the event source is active or passive.

An event provider working with an active event source typically registers itself with the event source as a listener and then waits to receive events. When an event arrives from the event source, the event provider submits the corresponding event data to the SQL-NS application, as illustrated in Figure 8.1.

Figure 8.1. An event provider working with an active event source.

[View full size image]

An event provider that works with a passive event source usually polls that event source periodically, looking for new events. If it finds new events, the event provider submits the appropriate event data to the SQL-NS application, as shown in Figure 8.2. Some of the event provider's polling attempts may not yield any events. In these cases, it simply waits until the next polling interval and then looks again.

Figure 8.2. An event provider working with a passive event source.

[View full size image]

Event Batches

When an event provider submits events to a SQL-NS application, it always groups them into units called event batches. Until now, we've mostly been considering the processing of individual events, but the batch concept becomes important when implementing event providers. An event batch represents a group of events that logically go together. SQL-NS treats a batch of events as an atomic unit: Either all the events in the batch are processed, or none of the events in the batch are processed.

Note

Processing events in batches rather than one at a time gives a performance advantage. The overhead involved in rule execution is incurred only once for the whole batch, instead of once per event. Batching does introduce latency because the entire batch must be submitted before its first event is processed. However, in most cases, the efficiency gains from batching lead to higher overall throughput than one at a time evaluation.

Earlier, when you looked at the SQL-NS application database objects, you saw that for each event class, the SQL-NS compiler created an events table named NS<EventClass>Events. You may also have noticed (although it wasn't explicitly pointed out) that there is also a table named NS<EventClass>EventBatches. As its name suggests, this table is used to track information about event batches. Each row in the table represents a single event batch and has columns for the event batch ID (a unique identifier for the event batch), the ID of the event provider that submitted the batch, as well as the times that the collection of the event batch started and ended. The events table relates to the event batches table via the EventBatchId column, which contains the ID of the batch that each event belongs to. Figure 8.3 illustrates this relationship.

Figure 8.3. The event batches and events tables.

[View full size image]

Each of the SQL-NS built-in event providers groups events into batches in different (although well-defined) ways. When building your own event providers, you can define any batching behavior you choose.

Event Provider Security

Because event providers connect between SQL-NS applications and the outside world, it's important to make sure that they are secure. Hackers must not be able to gain unauthorized access to the internal data of a SQL-NS application by means of an event provider and, perhaps more importantly, should not be able to compromise an event source by means of an event provider.

This section describes some of the facilities that SQL-NS offers for securing event providers. Note that these are just mechanisms. It's up to you to use them appropriately to secure the event providers in your applications.

Database Roles

As you've already seen, SQL-NS database security is role based. When the instance and application database objects are created, SQL-NS creates several database roles, each of which corresponds to a specific function of a running SQL-NS instance. It then sets the permissions on the various database objects so that only members of the appropriate roles can access them. For example, the permissions on the database objects involved in subscription management are set so that only members of the subscription management role are allowed access. You control the access that accounts have to the instance and application database objects by selectively granting them membership in the various SQL-NS roles.

NSEventProvider is the SQL-NS role for all operations related to event submission. Members of this role have access to all the database objects involved in event submission, including the events and event batches tables, as well as several internal tables, views, and stored procedures used by the SQL-NS engine.

If your event provider is hosted in the SQL-NS engine, it will run under the engine's account. If your event provider is standalone, it will most likely run under the account that started the standalone process. Event providers can connect to the SQL-NS database via either Windows Authentication (using the credentials of the Windows account under which they run) or SQL Server Authentication (using a SQL username and password). Regardless of which authentication scheme is used, the account with which the event provider connects to the database needs to be a member of the NSEventProvider role. You will need to grant membership in this role to the appropriate accounts when the application is deployed.

By using a low-privileged account to run your event provider and granting it only the permissions it needs (through role membership), you minimize the damage that a hacker could do if your event provider is compromised. Because membership in NSEventProvider does not provide access beyond the objects involved in event submission, the access that a hacker may obtain by hijacking the event provider is limited to these objects.

In Chapter 4, "Working with SQL-NS Instances," we examined the grant_permissions.cmd script that adds the service account to the NSRunService role (see the "Granting Permissions" section of Chapter 4, p. 92). NSRunService can be thought of as a combination of several separate roles, typically required by components running in the SQL-NS engine. Using NSRunService is a convenience when you have several components running under a single account. Instead of granting that account membership in all the separate roles, you just grant it membership in NSRunService. NSEventProvider is included in the list of roles that NSRunService covers. As in previous chapters, the SQL-NS service account is granted membership in NSRunService, so we don't explicitly need to grant it membership in NSEventProvider for its hosted event providers to work.

Later in this chapter, we'll build and test several standalone event providers. Because these are not hosted in the Windows service, they don't run under the service account. In the development stage, you'll run these using your development account, which doesn't need to be granted membership in NSEventProvider because it is already a system administrator. If you deployed any of these standalone event providers in a real production environment, you would have to grant membership in the NSEventProvider role to the account with which they connect to the database.

Argument Encryption

As shown later in this chapter, when you configure a hosted event provider in the ADF (in a <HostedProvider> element), you provide a set of arguments that the engine later passes to the event provider when it starts. When you compile your application, the compiler stores these arguments in a table in the application's database.

In some cases, these arguments may contain sensitive information, such as usernames and passwords used to connect to event sources. Simply storing these in the database in plain-text may present a security risk. For this reason, SQL-NS provides argument encryption: a facility for encrypting the event provider arguments before the compiler stores them in the database. The SQL-NS engine stores the encryption key securely in the Registry and uses it to decrypt the arguments before passing them to the event provider. Using this mechanism, you can greatly reduce the chance that an unauthorized user looking at the contents of your database will be able to obtain any sensitive information stored in arguments.

To configure argument encryption, you use the <EncryptArguments> element of the ICF. This is a Boolean element, and its value indicates whether to encrypt arguments. In previous chapters, the ICFs we've used have not specified the <EncryptArguments> element, which is equivalent to specifying the element with a false value. In this chapter, we add the <EncryptArguments> element to the ICF and set its value to true. A fragment of the ICF showing the <EncryptArguments> element is shown in Listing 8.1.

Listing 8.1. The <EncryptArguments> Element of the ICF

<NotificationServicesInstance>
  ...
  <EncryptArguments>true</EncryptArguments>
</NotificationServicesInstance>

Caution

You cannot change the value of the <EncryptArguments> element after the instance is created. The SQL-NS update tools return an error if they detect that the value of this element has been altered in the ICF when updating an instance. If you must change the value of <EncryptArguments>, you have to delete and then re-create the instance.

If you specify TRue for <EncryptArguments> in the ICF, you must provide an encryption key when you create, register, and update the instance. This key is used to encrypt and decrypt the arguments and is commonly referred to as the argument key. The argument key is used differently by the three operations:

  • When creating an instance, the SQL-NS compiler uses the argument key to encrypt the arguments specified in the ADF and store them in the database.

  • Registering an instance stores the argument key securely in the Registry so that the SQL-NS engine can obtain it when it needs to decrypt the arguments at runtime.

  • When updating an instance, the SQL-NS compiler uses the argument key to first decrypt the arguments currently stored in the database (so that it can compare them to the values in the ADF to detect changes) and then encrypt any new or modified argument values that it writes back to the database.

The argument key can be any string you choose. Both sets of SQL-NS tools (the nscontrol commands and the graphical tools in Management Studio) provide mechanisms for you to specify the key value. In the next section, we'll look at how these are used (see "Changes to the Scripts for Argument Encryption," p. 240).

Note

Argument encryption applies to all arguments supplied in the ADF and ICF, not just those for event providers. Other places that arguments are used include the configuration of delivery channels and content formatters. We'll look at these in more detail in Chapters 9, "Content Formatters," and 10, "Delivery Protocols."

To view the argument key used to encrypt the arguments of a SQL-NS instance, you can use the nscontrol displayargumentkey command or view the Encryption tab of the instance's properties dialog box in Management Studio (right-click the instance in the Notification Services folder in the Object Explorer and choose Properties). The account used to view the argument key must be a member of the Administrators or Power Users groups; otherwise, access to the argument key will be denied.


Preparing to Work with the Source Code in This Chapter

In this chapter, we continue working with the music store application that we've been developing over the past several chapters. We extend it with several new event providers that illustrate the various aspects of event provider development in SQL-NS.

This section describes the layout of the source files containing this chapter's code and the steps you need to take to get ready to use the working examples.

Source Files

As in previous chapters, we're going to modify the source code for the music store application in the C:\SQL-NS\Samples\MusicStore directory. Supplemental source files containing the code we look at in this chapter are provided in the C:\SQL-NS\Chapters\08\SupplementaryFiles directory. Table 8.1 provides a summary of the contents of these supplementary files.

Table 8.1. Supplementary Files and the Code They Contain

Filename

Code Added

Sections

InstanceConfiguration-0.xml

The ICF in its existing form, as first created in Chapter 5 (no code added)

Not used in this chapter; included only for reference

InstanceConfiguration-1.xml

The <EncryptArguments> element and a parameter definition used later in one of the event provider configurations

"Starting from a Clean Instance" (p. 241)

ApplicationDefinition-7.xml

FileSystemWatcherProvider declaration

"The FileSystemWatcherProvider" (p. 246)

ApplicationDefinition-8.xml

SQLProvider declaration and supporting event chronicle

"The SQLProvider" (p. 253)

ApplicationDefinition-9.xml

Continuous, custom, hosted event provider declaration

"Building a Continuous Custom Hosted Event Provider" (p. 269)

ApplicationDefinition-10.xml

Scheduled, custom, hosted, event provider declaration

"Building a Scheduled Custom Hosted Event Provider" (p. 281)

ApplicationDefinition-11.xml

Standalone event provider declarations

"Building Standalone Event Providers" (p. 291)

As before, the actual ICF and ADF that the compiler reads are C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml and C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml. In this chapter, when you're instructed to add code, you can either edit these files manually, or you can copy over one of the supplemental files. You then update the instance as you did in previous chapters to compile the new code.

The source files for the custom event providers that we build in this chapter are in the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents directory. We examine these files in the sections "Building Custom Hosted Event Providers" (p. 264) and "Building Standalone Event Providers" (p. 291).

Changes to the Scripts for Argument Encryption

As in previous chapters, we'll continue using batch files in the Scripts subdirectory under C:\SQL-NS\Samples\MusicStore to create, manipulate, and configure the SQL-NS instance. Many of these are the same as the ones you've seen in previous chapters, but some new ones are introduced to accommodate argument encryption, which we use in this chapter.

As explained in the "Argument Encryption" section (p. 237), you enable argument encryption by setting the value of the <EncryptArguments> element in the ICF to true. Listing 8.1 showed the fragment of the ICF we'll use in this chapter that sets this element. When we invoke the nscontrol create, register, and update commands, we need to pass the argument key value. This is done by means of the -argumentkey parameter, which each of these commands takes.

In the Scripts directory, you'll find scripts called create_with_argument_key.cmd, register_with_argument_key.cmd, and update_with_argument_key.cmd. These are equivalent to the scripts you used in previous chapters (create.cmd, register.cmd, and update.cmd, respectively) except that the nscontrol commands in the new scripts are invoked with the -argumentkey parameter. As an example, let's look at the call to nscontrol create in create_with_argument_key.cmd:

[View full width]

nscontrol create -in "%INSTANCE_BASE_DIRECTORY_PATH%\InstanceConfiguration.xml"
  %SQL_USER_INFO% -argumentkey MusicStoreArgumentKey _SqlServer_=%SQL_SERVER% _NSServer_=
  %NS_SERVER% _InstanceBaseDirectoryPath_="%INSTANCE_BASE_DIRECTORY_PATH%"

The boldface portion of the command supplies the string, MusicStoreArgumentKey, as the value of the -argumentkey parameter. You can use any string value for the argument key; I've chosen the rather unimaginative value you see here purely for simplicity. Take a look at the other two scripts to see how the -argumentkey parameter is used with nscontrol register and nscontrol update.

Note

Whatever argument key you decide to use, it's important that you pass the same value for the -argumentkey parameter to nscontrol create, register, and update.

The steps in the following section instruct you to use create_with_argument_key.cmd to create the music store instance from scratch and register_with_argument_key.cmd to register it. When you update the instance to add the new code for this chapter, make sure you use update_with_argument_key.cmd instead of update.cmd. If you use the wrong script, the argument key won't be supplied to nscontrol update, resulting in a compile error.

Specifying an Argument Key When Creating, Registering, and Updating an Instance with the Management Studio Tools

Like the command-line tools, the SQL-NS tools in Management Studio also provide a way for you to specify an argument key when creating, registering, and updating an instance that uses argument encryption. The dialog boxes used to invoke these commands each have a page labeled Encryption, which contains a text box you can use to enter the argument key value.

You can select the Encryption page from the page selector in the left side panel of the New Notification Services Instance, Register Instance, and Update Instance dialog boxes. In all of these dialog boxes, enter the argument key in the text box on the Encryption page before clicking the OK button to invoke the command.

Starting from a Clean Instance

Although this chapter extends the prototype of the music store application that you built in previous chapters, you will need to recompile the music store instance from scratch, rather than just update the version you created earlier. Besides ensuring that you're starting from a clean instance (free of any artifacts that may have developed from earlier testing and debugging), this is necessary because we are now using argument encryption. Recall that it's not possible to update an existing instance to turn on argument encryption: You must first delete and then re-create the instance.

Before starting the work of this chapter, perform the following instructions to establish a clean instance:

1.
From a Notification Services Command Prompt on your development machine, navigate to the music store instance's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run cleanup.cmd to remove the music store instance and all associated files that might have been left over from previous chapters' work.

3.
Copy the supplementary file InstanceConfiguration-1.xml from this chapter's supplementary file directory over the existing music store ICF using the following command:

copy /y C:\SQL-NS\Chapters\08\SupplementaryFiles\InstanceConfiguration-1.xml
C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml

This changes the ICF to include the <EncryptArguments> element (set to true, as shown in Listing 8.1) and adds one parameter definition that we'll need later.

4.
Make sure that the music store ADF, C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml, contains the complete code for the music store prototype, as developed so far. It should contain the same code as the supplementary file ApplicationDefinition-6.xml in the Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State," supplementary file directory, C:\SQL-NS\Chapters\06\SupplementaryFiles. If you completed the steps in previous chapters, the ADF should already have the correct contents. Otherwise, you can copy the supplementary ADF now.

5.
Run create_music_store_database.cmd. This rebuilds the music store database.

6.
Run create_with_argument_key.cmd to compile the instance and application from scratch.

7.
Run register_with_argument_key.cmd to register the instance.

8.
Run grant_permissions.cmd to assign the appropriate database permissions.

9.
Run enable.cmd to enable the instance.

Caution

If you accidentally invoked register.cmd instead of register_with_argument_key.cmd in step 7, the argument key will not be written to the Registry. The call to nscontrol register in register.cmd does not specify an argument key value (the -argumentkey parameter is omitted entirely).

As a result, the SQL-NS service will fail to start. Because the instance uses argument encryption, the service cannot do any processing if it can't retrieve the argument key from the Registry. If you use the net start command to start the service, you'll see an error message that reads, A system error has occurred. Also, an event message will be written to the Application Event Log. To see this message, open the Event Viewer from Administrative Tools, select the Application log, and look for a message with NotificationServices as the event source name and an event ID of 6. The description in the message will read:

The argument key was not provided but an argument key was given when the
database was created.

To correct the situation, first unregister the instance using the unregister.cmd script (also located in the C:\SQL-NS\Samples\MusicStore\Scripts directory). Then run register_with_argument_key.cmd.

You do not need to add subscribers and subscriptions. Because we're focusing on event submission, all the tests in this chapter go only as far as making sure that the correct events end up in the events table. The tests do not check how those events are matched with subscriptions. We can do this with confidence because we've already tested the matching capabilities of the application in previous chapters. We know that when events end up in the events table, the matching logic in the application works. This chapter just presents several alternative ways to get events into the events table. It is a testament to the elegant design of the SQL-NS architecture that this kind of modular testing is possible.

The AddSongs Program

In the music store application, events represent songs added to the music store's catalog. In previous chapters, we simulated events by inserting the IDs of songs already in the music store catalog into the event class view. We never actually added any new songs to the catalog.

Some event providers in this chapter do look at the data in the music store catalog to determine whether new songs have been added. To test these event providers, we need a way to add new song records to the catalog tables. I've provided a program called AddSongs that presents a simple user interface for doing this. A Visual Studio solution containing the source code for this program is in the C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs directory. Before continuing, build this solution so that you can run the program when it's needed later:

1. | Open the solution file, C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs\AddSongs.sln, in Visual Studio.
2. | Build the solution. The project should compile with no errors.

At this point, it's not necessary for you to go through the source code for the AddSongs program. The "Submitting Events from an Application" section (p. 296) later in this chapter describes how this program can act as a standalone event provider; in that section, we look at the parts of its source code involved in event submission. For now, just start the program in Visual Studio (by going to the Debug menu and selecting Start Debugging, or by pressing F5) so that you can learn how to use it.

Caution

For AddSongs to work correctly, the music store database must already have been created. Before running the AddSongs program, make sure that you've completed the instructions in the "Starting from a Clean Instance" section (p. 241).

When AddSongs starts, it opens a database connection dialog box in which you specify how to connect to the music store database. Enter the name of the SQL Server hosting the database and select either Windows Authentication or SQL Server Authentication. If you select SQL Server Authentication, supply the username and password of the development account you created in Chapter 2, "Getting Set Up." Click OK to connect to the database.

After AddSongs connects to the database, you should see a window like the one shown in Figure 8.4.

Figure 8.4. The AddSongs program.

[View full size image]

AddSongs allows you to enter a new album, with up to five songs. It's obviously a rather limited application. It does not allow you to add songs to existing albums, browse the songs already in the database, or enter more than five songs. However, for the simple purpose of adding songs to the database to test event providers, it is sufficient.

Enter an album title and artist name into the first two text boxes and pick a genre from the drop-down list. You can then enter up to five songs and click Add to Database to create the album and song records in the music store catalog. For now, leave the Submit Events for Songs Added box unchecked. I'll explain the use of this check box later in this chapter.

Take the time to play with AddSongs and familiarize yourself with it. Later in this chapter, you'll be asked to use it to add songs to the database.

Note

After having built the AddSongs program as instructed in this section, you can start it later by invoking the AddSongs.exe executable directly from the solution's binaries directory, C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs\bin\Debug.

Installing Message Queuing

One of the event provider examples in this chapter uses Message Queuing, an optional Windows component. If you don't have this component installed, you should install it now. If you choose not to install Message Queuing, you will still be able to work through most of the material in this chapter. You just won't be able to run the continuous hosted event provider described later.

To install Message Queuing, use the following instructions:

1.
From your Start menu, open Control Panel and select Add or Remove Programs.

2.
In the left pane of the Add or Remove Programs window, choose the Add or Remove Windows Components icon. The Windows Components Wizard opens.

3.
Find the Message Queuing item in the components list. On some versions of Windows, Message Queuing is not in the top-level components list. If you don't see it in the first list in the Windows Components Wizard, select the Application Server item (which should be in the first list) and click the Details button. Message Queuing should be an item in the subcomponents dialog box that opens.

4.
Check the box next to the Message Queuing item, as shown in Figure 8.5. If the box is already checked, you already have Message Queuing installed. You can cancel the wizard and skip the rest of these instructions.

Figure 8.5. The Windows Components Wizard.

[View full size image]

5.
If your development machine is not part of a domain, you must not install the Active Directory Integration subcomponent of Message Queuing: select the Message Queuing component, click the Details button, and deselect the Active Directory Integration option in the subcomponents dialog box. If your development machine is part of a domain, you can skip this step (the default subcomponents of Message Queuing will be installed).

6.
Close any sub-component dialog boxes that may be open and then click the Next button in the wizard to allow Windows Setup to install the necessary files (you may be prompted to insert your Windows installation CDs).

7.
Finish the wizard, close Add or Remove Programs, and close the main Control Panel window.


Using the Built-in Event Providers

SQL-NS provides three built-in event providers:

  • The FileSystemWatcherProvider An event provider that reads event data from XML files in the filesystem

  • The SQLProvider An event provider that runs a SQL query on a periodic interval to obtain event data

  • The AnalysisServicesProvider An event provider that runs dynamic multidimensional (MDX) queries to obtain event data from a cube.

All three are hosted event providers, so they run inside the SQL-NS engine. To use them in an application, you simply configure them in the ADF. When the SQL-NS engine starts, it reads the configuration information and loads the required event provider components.

This section covers two of the three SQL-NS built-in event providers: the FileSystemWatcherProvider and the SQLProvider. We look at what these event providers do and how to configure them for use in SQL-NS applications.

The third event provider, the AnalysisServicesProvider, is not covered here because its use involves multidimensional analysis, which is beyond the scope of this book. To learn about the AnalysisServicesProvider, consult the SQL-NS Books Online, which provides complete reference documentation, including code samples.

The FileSystemWatcherProvider

The FileSystemWatcherProvider works by monitoring a directory in the filesystem; when it sees a new XML file added to that directory, it opens the file and submits the event data in it. The FileSystemWatcherProvider is useful in cases where event data can be written to XML files either by the event source directly or by an intermediate process.

To use the FileSystemWatcherProvider, you configure it in a <HostedProvider> element in the ADF. Listing 8.2 shows the <HostedProvider> declaration we can add to the music store application's ADF to configure it to use the FileSystemWatcherProvider.

Listing 8.2. The FileSystemWatcherProvider Declaration in the ADF

<Application>
...
  <Providers>
    <HostedProvider>
      <ProviderName>SongAddedFSWProvider</ProviderName>
      <ClassName>FileSystemWatcherProvider</ClassName>
      <SystemName>%_NSServer_%</SystemName>
      <Arguments>
        <Argument>
          <Name>WatchDirectory</Name>
          <Value>%_ApplicationBaseDirectoryPath_%\EventsWatchDirectory</Value>
        </Argument>
        <Argument>
          <Name>SchemaFile</Name>
          <Value>%_ApplicationBaseDirectoryPath_%\SongAddedEventSchema.xsd</Value>
        </Argument>
        <Argument>
          <Name>EventClassName</Name>
          <Value>SongAdded</Value>
        </Argument>
      </Arguments>
    </HostedProvider>
    ...
  </Providers>
...
</Application>

Within the <HostedProvider> element, the <ProviderName> element assigns a "friendly name" to the event provider that can be used for configuration, administration, and reporting purposes after the application is created. The <ClassName> indicates the event provider type (or class); in this case, FileSystemWatcherProvider. You can have many instances of the FileSystemWatcherProvider in a single application (each perhaps monitoring a different directory in the filesystem). For each instance, you would need to provide a <HostedProvider> declaration that specifies FileSystemWatcherProvider as the class name, but provides a unique provider name. The key point is that FileSystemWatcherProvider is the event provider type, not the event provider name.

The <SystemName> element specifies the machine on which the event provider should run. If you're using the Standard Edition of SQL-NS, all the components of your application must run on the same machine, but if you're using Enterprise Edition, you can configure components to run on different machines.

The <Arguments> element supplies values for the three arguments required by the FileSystemWatcherProvider:

  • WatchDirectory specifies the directory in the filesystem that the FileSystemWatcherProvider should monitor.

  • SchemaFile specifies the full path to an XSD schema file that describes the data in the event files that will be dropped in the watch directory.

  • EventClassName specifies the event class of the events the provider will submit (the files dropped in the watch directory must contain data corresponding to the fields of this event class).

These three arguments are required by the FileSystemWatcherProvider. If you don't supply them, the event provider will fail to start.

In the case of the event provider configuration shown in Listing 8.2, the watch directory specified is a subdirectory off the application's base directory. The event class name specified is SongAdded, the name of the only event class in the music store application. Based on this declaration, the event provider will expect to submit SongAdded events, so the event files that it picks up from the watch directory must contain SongAdded event data. The schema file specified describes the structure of the XML data in event files.

Event Files and Event File Schemas

The event files processed by the FileSystemWatcherProvider contain event data in XML form. In this section, we look at a sample event file containing a few SongAdded events and examine its structure. When we test the FileSystemWatcherProvider later in this section, we'll use this event file. On your system, you'll find it at C:\SQL-NS\Chapters\08\DataFiles\EventData.xml. Listing 8.3 shows the contents of this file.

Listing 8.3. A Sample Event File Used by the FileSystemWatcherProvider

<eventData>
    <SongAdded>
        <SongId>1</SongId>
    </SongAdded>
    <SongAdded>
        <SongId>6</SongId>
    </SongAdded>
    <SongAdded>
        <SongId>12</SongId>
    </SongAdded>
    <SongAdded>
        <SongId>14</SongId>
    </SongAdded>
</eventData>

The root element is <eventData> (note the lowercase "e"). The FileSystemWatcherProvider requires that all event files have <eventData> as the root element, but the contents within <eventData> vary per event class. There must be one XML element within <eventData> for each event to be submitted, and the structure of this element must be defined in the schema file referenced in the SchemaFile event provider argument.

In the example shown in Listing 8.3, each SongAdded event is defined in a <SongAdded> element. This element has a subelement for the SongId event field, called <SongId>. The outer element that encapsulates the event can have any name you choose. It's just a convention to give this element the same name as the event class, as this example does. However, the subelements of the event must be named the same as the event class fields that they correspond to.

The schema file referenced in the SchemaFile event provider argument in Listing 8.2 describes the structure of the <SongAdded> element in the event data file. The purpose of this schema file is to tell the FileSystemWatcherProvider how to map the XML definition of an event to the event fields defined in the ADF. You can find this schema file on your system at C:\SQL-NS\Samples\MusicStore\SongAlerts\SongAddedEventSchema.xsd. Listing 8.4 shows its contents.

Listing 8.4. Schema File Used by the FileSystemWatcherProvider to Process SongAdded Events

[View full width]
<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn
  :schemas-microsoft-com:mapping-schema" >
    <xsd:element name="SongAdded" sql:relation="SongAdded" >
        <xsd:complexType>
            <xsd:sequence>
                <xsd:element name="SongId" type="xsd:integer" />
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>
</xsd:schema>

This schema is defined in XSD (a standard for XML schemas) and annotated with attributes defined by SQLXML. These attributes provide the mapping between the elements of the XML and SQL data. Schemas with these SQLXML attributes are called SQL-annotated schemas. This section covers only the parts of the schema that are relevant in the context of the FileSystemWatcherProvider. For complete information on SQL-annotated schemas, see the "SQLXML" section in the Microsoft MSDN Library at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnanchor/html/anch_SQLXML.asp.

The schema shown in Listing 8.4 defines the structure of the <SongAdded> element that we saw in Listing 8.3. Note the use of the sql:relation attribute that specifies the name of the event class. Whenever you define a schema for event data used with the FileSystemWatcherProvider, you must provide this attribute on the root element, with a value that specifies the event class name.

Caution

The event schema file does not define the outermost <eventData> element in the event file. All event files must have <eventData> as the root element (like the one in Listing 8.3 does). The schema file just defines the structure of the elements that appear within <eventData>.

The schema defines a subelement for each field in the event class. Each subelement must have the same name as its corresponding event field and its declared data type must be equivalent to the event field's SQL data type (as specified in the <FieldType> element in the event class declaration in the ADF). In Listing 8.4, the schema defines an element for the SongId event field and specifies its type as xsd:integer.

The Runtime Behavior of the FileSystemWatcherProvider

When the FileSystemWatcherProvider starts, the SQL-NS engine passes it the argument values specified in the ADF. The FileSystemWatcherProvider validates these arguments and then begins monitoring the specified directory.

Any XML files in the watch directory that have not been processed at the time that the event provider starts are processed immediately. The FileSystemWatcherProvider opens these files, reads the XML data in them, and submits the corresponding event data to the SQL-NS application. It then waits for new files to appear in the watch directory. Whenever a new file appears, the FileSystemWatcherProvider processes it immediately.

All the events in a single XML event file are treated as a batch. Thus, the FileSystemWatcherProvider will create one event batch for each file it processes.

When the FileSystemWatcherProvider successfully processes a file, it renames that file according to the following pattern:

<OriginalFileName>.<Timestamp>.done

<OriginalFileName> represents the original name of the file; <Timestamp> is a timestamp that indicates when the file was processed. For example, if the original filename was Eventdata.xml, after processing, the filename might be

EventData.xml.20040429-102956.31.done

If the FileSystemWatcherProvider encounters an error while processing a file, it renames that file with a .err extension. Files with this extension are not picked up again.

Tip

If the FileSystemWatcherProvider encounters errors, it will log detailed error messages to the Application Event Log. To see these messages, open the Event Viewer from Administrative Tools and look for messages in the Application log with NotificationServices as the event source name.

Note

Note that the FileSystemWatcherProvider only monitors for new file additions to the watch directory. It does not detect changes to existing files within that directory.

Permissions Required by the FileSystemWatcherProvider

As described in the "Event Provider Security" section (p. 236) earlier in this chapter, membership in the NSEventProvider database role is required to perform event submission operations in a SQL-NS database. In the case of the FileSystemWatcherProvider, filesystem permissions are also needed. Specifically, the Windows account under which the FileSystemWatcherProvider runs (usually the SQL-NS service account) needs both read and write permissions to the events watch directory. Write permissions are needed because the FileSystemWatcher renames the event files after it processes them (as described in the previous section).

Permissions on the events watch directory for the music store sample application are granted by the script C:\SQL-NS\Chapters\08\Scripts\setup.cmd, which was run during the setup steps when you first configured your development environment. When you create your own applications, you'll need to grant read and write permissions on any events watch directories to the service account yourself.

Tip

If the FileSystemWatcherProvider does not have sufficient privileges to read from or write to the events watch directory, event data will not be submitted to the SQL-NS application and the event files will not be renamed. If this occurs, it may appear that the FileSystemWatcherProvider is not doing anything. If you see this behavior, check the Application Event Log for error messages from the FileSystemWatcherProvider.

Testing the FileSystemWatcherProvider in the Music Store

Application In this section, we add the FileSystemWatcherProvider configuration (shown in Listing 8.2) to the music store application and then test it with the sample event file shown in Listing 8.3. The instructions in this section assume you've completed the steps in the "Starting from a Clean Instance" section (p. 241) earlier in this chapter. If you did not complete those steps earlier, do so now before proceeding.

Complete the following steps to get the music store application running with the FileSystemWatcherProvider:

1.
Add the code shown in Listing 8.2 to the music store application's ADF (C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml). You can either enter this code manually or copy over the supplementary ADF, ApplicationDefinition-7.xml, from the Chapter 8, "Event Providers," supplementary files directory, C:\SQL-NS\Chapters\08\SupplementaryFiles.

2.
From a Notification Services Command Prompt on your development machine, navigate to the music store sample's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

3.
Run disable.cmd.

4.
If the instance's Windows service is running, stop it by typing the following command:

net stop NS$MusicStore

5.
Run update_with_argument_key.cmd to update the instance. As before, nscontrol update summarizes the changes it will apply to the database and prompts for confirmation. Respond with y to accept the changes.

6.
Run enable.cmd.

7.
Start the instance by typing the following command:

net start NS$MusicStore

Note

Steps 2 to 7 are almost the same as the steps you used in previous chapters to update the music store instance. The only difference is that the script update_with_argument_key.cmd is used, instead of update.cmd. This is required because the instance now uses argument encryption. From now on, whenever you're instructed to update the instance, you should use these steps.

If you accidentally invoke the update.cmd script (which does not supply the -argumentkey parameter to nscontrol update) instead of update_with_argument_key.cmd, you'll receive an error message that reads:

The EncryptArguments value is true; you must provide an argument key when you
create or update the instance.

The output from nscontrol enable (invoked in step 6 of the preceding instructions) should now list the new FileSystemWatcherProvider event provider as one of the components of the application. Using the following steps, copy an event data file into the directory being monitored by the FileSystemWatcherProvider to see it in action:

1.
From a Notification Services Command Prompt on your development machine, copy the sample event data file into the events watch directory with the following command:

copy C:\SQL-NS\Chapters\08\DataFiles\EventData.xml C:\SQL-NS\Samples\
MusicStore\SongAlerts\EventsWatchDirectory

2.
Look at the contents of the events watch directory by executing the following command:

dir C:\SQL-NS\Samples\MusicStore\SongAlerts\EventsWatchDirectory

Within a few seconds, you should see the event file you copied in (Eventdata.xml) renamed to the form Eventdata.xml. .<Timestamp>.done.

3.
In Management Studio, execute the following query to see the data in the events table:

USE MusicStore
SELECT * FROM [SongAlerts].[NSSongAddedEvents]

In the event rows returned by the query in the last step, you should see the song IDs that were specified in the events file (shown in Listing 8.3). Also, notice that all the event rows have the same event batch ID. This is because the FileSystemWatcherProvider submits all the events in a file as a single batch.

The SQLProvider

The SQLProvider is the second built-in event provider that SQL-NS offers. Its purpose is to provide an easy way to submit event data into SQL-NS applications from event sources in SQL databases.

Note

In the SQL-NS Books Online and other documents, the SQLProvider is sometimes referred to as the SQL Server Event Provider.

The SQLProvider works by periodically executing an application-defined SQL query at a specified interval. The query reads data from one or more tables that serve as event sources. Whatever data the query returns, the SQLProvider submits as a batch of events. The execution of the query and the submission of its results as events both happen in the database. Thus, the SQLProvider facilitates the transfer of the event data from the source tables to the SQL-NS application, without that data having to leave the database.

To see why this is useful, consider how events originate in the music store application. When a song is added to the music store's inventory, a record for that song is added to the songs table in the music store catalog. An event containing the ID of that song then needs to be submitted to the SQL-NS application. Using an event provider that operates outside the database (such as FileSystemWatcherProvider), the song ID would have to be read from the music store catalog in the database, copied into the event provider's memory, and then copied back into the database to be inserted into the SQL-NS application's events table. Using the SQLProvider eliminates the copying of the event data into the event provider's memory. The SQLProvider can be configured with a query that returns the IDs of the new songs in the music store catalog, and it will execute this query in such a way that the results are submitted directly to the SQL-NS application's events table.

Using the SQLProvider

The SQLProvider is a hosted event provider, so you declare it in a <HostedProvider> element in your ADF. In the declaration of the SQLProvider, you specify the query that the event provider should run. This is referred to as the events query. You also specify the interval period on which the events query should be invoked. When the event provider starts, it invokes the events query regularly on this interval and treats whatever data the query returns as a batch of events.

To use the SQLProvider with the music store application, we need to design an events query that returns the appropriate event data. This will be a query over the songs in the music store catalog that returns the IDs of the new songs. We can define "new songs" to mean songs added since the previous time the query was run. If we keep track of the time at which the events query last ran, we can determine which songs are new by looking at the times they were added to the database. (The music store database already tracks the time each song was added.) The query should return only the IDs of songs added after the last time the events query ran.

Keeping track of the last time at which the events query was run requires a chronicle. Recall that a chronicle is a SQL table that can be used to keep application state. To support the events query for the SQL provider, we need to add a new chronicle to the music store ADF. Listing 8.5 shows this chronicle declaration.

Listing 8.5. The SQLProviderChronicle Declaration in the ADF

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      ...
      <Chronicles>
        ...
        <Chronicle>
          <ChronicleName>SQLProviderChronicle</ChronicleName>
          <SqlSchema>
            <SqlStatement>
              --Handle update by dropping the table
              --first if it exists.
              IF EXISTS (
                  SELECT so.name
                  FROM sys.objects so
                  JOIN sys.schemas sc ON so.schema_id = sc.schema_id
                  WHERE so.name = 'SQLProviderRunTime'
                      AND so.type = 'U'
                      AND sc.name = 'SongAlerts'
              )
                  DROP TABLE [SongAlerts].[SQLProviderRunTime]
            </SqlStatement>
            <SqlStatement>
              CREATE TABLE [SongAlerts].[SQLProviderRunTime]
              (
                  LastRunTime DATETIME NOT NULL
              )
            </SqlStatement>
            <SqlStatement>
              -- GRANT permissions on the table.
              GRANT SELECT ON [SongAlerts].[SQLProviderRunTime] TO [NSEventProvider]
              GRANT UPDATE ON [SongAlerts].[SQLProviderRunTime] TO [NSEventProvider]

              GRANT SELECT ON [SongAlerts].[SQLProviderRunTime] TO [NSRunService]
              GRANT UPDATE ON [SongAlerts].[SQLProviderRunTime] TO [NSRunService]
            </SqlStatement>
            <SqlStatement>
              INSERT INTO [SongAlerts].[SQLProviderRunTime](LastRunTime)
              VALUES (GETUTCDATE())
            </SqlStatement>
          </SqlSchema>
        </Chronicle>
      </Chronicles>
      ...
    </EventClass>
  </EventClasses>
  ...
</Application>

This chronicle consists of a table with a single column, LastRunTime, that holds the last time at which the SQLProvider's events query was run. The first <SqlStatement> in the chronicle definition drops the table if it already exists. (As in previous examples, this is needed to handle updating the instance.) The second <SqlStatement> creates the table.

The next <SqlStatement> grants SELECT and UPDATE permissions on the chronicle table to the NSEventProvider and NSRunService roles. This is needed because for security reasons, the events query runs in a low-privilege context. The user running the events query (this is always the account with which the SQL-NS engine connects to the database) must be explicitly granted any permissions needed by the events query. As you'll see shortly, our events query both selects from and updates the chronicle table, so SELECT and UPDATE permissions are needed. Because the SQL-NS engine's account will always either be a member of NSRunService (the consolidated role that covers all operations handled by the engine) or NSEventProvider (the role specific to event submission), it's sufficient to grant these roles the needed permissions.

The final <SqlStatement> in the chronicle definition initializes the table with the current UTC time. This ensures that on the first execution of the events query, it picks up only the songs added after the chronicle was created.

Notice that we have not defined any event chronicle rules for this chronicle. Even though this chronicle is defined as part of the SongAdded event class, it is not updated by event-triggered chronicle rules. Rather, this chronicle will be updated as part of the events query that the SQLProvider runs.

Let's take a look at the <HostedProvider> declaration for the SQLProvider, which includes the definition of the events query. This is shown in Listing 8.6.

Listing 8.6. The SQLProvider Declaration in the ADF

<Application>
  ...
  <Providers>
    ...
    <HostedProvider>
      <ProviderName>SongAddedSQLProvider</ProviderName>
      <ClassName>SQLProvider</ClassName>
      <SystemName>%_NSServer_%</SystemName>
      <Schedule>
        <Interval>P0DT00H00M30S</Interval>
      </Schedule>
      <Arguments>
        <Argument>
          <Name>EventsQuery</Name>
          <Value>
            -- Declare variables used to store the last and current
            -- run times.
            DECLARE @lastRunTime    DATETIME
            DECLARE @currentRunTime DATETIME

            -- Select the last run time from the chronicle table.
            SELECT  @lastRunTime = LastRunTime
            FROM    [SongAlerts].[SQLProviderRunTime]

            -- Obtain the current run time.
            SELECT  @currentRunTime = GETUTCDATE()

            -- Update the chronicle, setting the current run time
            -- as the last run time.
            UPDATE  [SongAlerts].[SQLProviderRunTime]
            SET     LastRunTime = @currentRunTime

            -- Select all the songs added between the last run time
            -- and the current run time.
            SELECT  sd.SongId
            FROM    [Catalog].[SongDetails] sd
            WHERE   sd.DateAdded &gt; @lastRunTime
            AND sd.DateAdded &lt;= @currentRunTime
          </Value>
        </Argument>
        <Argument>
          <Name>EventClassName</Name>
          <Value>SongAdded</Value>
        </Argument>
      </Arguments>
    </HostedProvider>
    ...
  </Providers>
  ...
</Application>

This declaration assigns a provider name and specifies the provider class as SQLProvider. As with previous event provider declarations you've seen, the <SystemName> element specifies the machine on which the event provider should run.

The <Schedule> element specifies the schedule on which the events query should be invoked. The <Schedule> element can actually have two subelements, <StartTime> and <Interval>, although only <Interval> is shown here. (<StartTime> is optional.)

Note

The <StartTime> element often causes confusion, and I've seldom seen it used in real applications. It does not specify a start time for the events query, as you might assume from its name, but it is used to calculate an offset from the event provider's start time that determines when the events query is first run. Refer to the SQL-NS Books Online for more information about this element.

The <Interval> element specifies the periodic interval on which the events query is run. The value of the <Interval> element is specified in the syntax of the XML duration data type. This syntax is explained in the sidebar "The XSD duration Data Type" (p. 157) in Chapter 5. In Listing 8.6, the interval value is given as

P0DT00H00M30S

This specifies a period of 30 seconds (the days, hours, and minutes are zero). The year and month components are omitted entirely. (It's legal to omit components that have zero values.)

Note

The <Schedule> element is not specific to the SQLProvider. It is part of the general <HostedProvider> element and is used to specify a schedule for any scheduled, hosted event provider. This same <Schedule> element is used when we configure the custom, scheduled, hosted event provider described in the "Building a Scheduled Custom Hosted Event Provider" section (p. 281).

The <Arguments> element in Listing 8.6 specifies the events query and the event class of the events that the provider will submit. The events query can be a single statement, or a batch of statements, as long as the result contains only a single rowset. In Listing 8.6, the events query is a batch of statements that ultimately selects a set of song IDs.

Note

The SQLProvider also accepts another argument: PostQuery, not shown in Listing 8.6. This argument specifies a query that is run after the events query, to do cleanup or update state information. In this example, the PostQuery is not needed, so it's omitted. The SQL-NS Books Online provide more information on the PostQuery argument.

The first part of the events query declares two local variables: one to hold the last time the events query was run, the other to hold the current time. The last run time value is read from the chronicle table, and the current time is obtained by calling the built-in SQL function GETUTCDATE(). The next statement updates the chronicle table, setting the last time the events query was run to the current time. Finally, the query selects the song IDs from the music store catalog's SongDetails view, where the timestamp is between the last and current runtimes.

The columns returned by the events query must exactly match the fields in the event class. In this case, the SongAdded event class contains only a SongId field, so the query just returns the song IDs of the new songs.

Although this particular events query is specific to the music store, the pattern is fairly common among queries used with the SQLProvider. Because the SQLProvider is periodic in nature, the events query is typically written to return data about events that have occurred between the last time the events query was run and the current time. The semantics of how the times are tracked, and how event data is filtered based on the time intervals, are specific to each application.

When writing this type of events query, you must be careful to avoid race conditions. Remember that while the query itself is executing, new data can arrive in the source tables from which the query reads. In the case of the music store application, this means that new song records could be added to the music store database in the middle of a particular invocation of the events query. It's important that this data be handled correctly. It should be processed, but only once, either during the current invocation of the query, or the next one. The data should never be skipped or processed during both the current and the next invocation.

In the events query shown in Listing 8.6, notice that the value for the current time is obtained once at the beginning of the query and stored in a local variable. This provides an unchanging reference point that can be used throughout the rest of the query. The query will return only the IDs of songs whose timestamps occur before this current time value. Even if songs are added while the query is executing, after the current timestamp is obtained, they will be processed by the next invocation of the query.

Testing the SQL Provider in the Music Store Application

To see the SQLProvider in action, you need to add the code in Listings 8.5 and 8.6 to the ADF. As with previous code additions, you can either manually type the code into C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml, or you can use the supplemental ADF, ApplicationDefinition-8.xml. After adding the code to your ADF, update the music store instance using the same steps you used before (for reference, see steps 27 in the "Testing the FileSystemWatcherProvider in the Music Store Application" section, p. 251).

The SQLProvider looks at the music store catalog to find new songs, so to test it, we need to add a few songs to the music store catalog using the AddSongs program described earlier. The event provider should pick up these new song records and submit them as events.

Use the following steps to test the SQLProvider:

1.
Start the AddSongs program that you built earlier (see "The AddSongs Program," p. 243).

2.
Enter album information and a few song titles (the values you use do not matter, as long as you don't enter songs or albums that are already in the databasejust use the information from your favorite album).

3.
Click the Add to Database button (make sure the Submit Events for Songs Added box is not checked). This writes the song records to the database.

4.
Execute the following query in Management Studio to see the song records in the music store catalog, including the ones you just added:

USE MusicStore
SELECT * FROM [Catalog].[SongDetails]

When the SQLProvider's events query runs, it will detect the new songs and submit their song IDs as events. Wait at least 30 seconds (recall that the SQLProvider runs the events query every 30 seconds) and then run the following query to see the event data in the events table:

USE MusicStore
SELECT * FROM [SongAlerts].[NSSongAddedEvents]

You should see rows in the events table containing the song IDs of each of the songs you just added.

If you look at the event batches table for the SongAdded event class ([SongAlerts]. [NSSongAddedEventBatches]), you should see several event batches with event counts of zero. These empty event batches exist because the SQLProvider creates an event batch every time it runs the events query, even if the query returns no data. Except for the extra space the batch records occupy in the event batches table, these empty event batches have no effect on the system: The generator simply ignores them.


APIs for Building Custom Event Providers

The previous section explained the use of two SQL-NS built-in event providers. The built-in event providers are convenient and flexible enough to use in most applications, just by supplying the required configuration information in the ADF. But if your application needs event provider capabilities beyond those available in the built-in event providers, you will need to implement a custom event provider.

Building a custom event provider involves writing code that implements the event submission operations. For example, the code for a custom event provider might read event data from an event source on the network, transform that data in some way, and then submit it as events to a SQL-NS application. The next two sections of this chapter cover custom event provider implementation in detail. In preparation for that material, this section explains the SQL-NS APIs used in event provider development. These APIs, called event submission APIs, offer convenient ways to transfer event data into the events tables in SQL-NS application databases.

Note

The event submission APIs are common to all the event provider forms. Regardless of whether you're building a hosted, scheduled event provider or a standalone event provider, you use the APIs described in this section to submit events to your SQL-NS application. For this reason, the APIs are described here independently of any particular event provider type. In the code samples later in this chapter, you'll see these APIs put to practical use in various kinds of event providers.

SQL-NS has three sets of event submission APIs, each of which provides a different way to submit events. This section is an overview that describes the facilities that the three APIs offer. Which API is appropriate for your event provider depends largely on the form in which you obtain event data from the event source. When you understand these APIs, you'll be able to determine which is the most natural fit for the particular event source and event data you're working with.

As with all other descriptions of APIs in this book, the information presented here is not intended to replace the exhaustive reference information provided in the SQL-NS Books Online. This book's examples show the most useful parts of the APIs and illustrate some typical usage patterns, but you should always consult the SQL-NS Books Online as the ultimate reference.

The Event Object API

As its name suggests, the Event Object API represents events as objects that the event provider code can manipulate. Event provider implementations can create event objects, set data properties on them, and then submit them to the SQL-NS application in batches. Internally, the Event Object API writes the data stored in the properties of the event objects to the events tables.

The Event Object API consists of two classes: Event and EventCollector. Typically, an event provider instantiates one event collector object (an instance of the EventCollector class), which acts as a gathering point for events. It creates Event objects to represent individual events and submits these to the event collector to form a batch. The event collector then writes the event data to the events table in the SQL-NS application database.

Note

It turns out that Event was a poor choice for the name of the class that represents events. When someone says "event class," it can be unclear whether he is referring to the class, Event, in the Event Object API, or the ADF concept of an event class (which defines a type of event that an application may receive). However, despite this apparent confusion, the meaning should be discernable from the context in which the term is used.

When you create an EventCollector object, you initialize it with an NSApplication object (that represents a connection to the SQL-NS application) and the event provider name. It is then ready to start collecting events.

To submit a single event, you create an Event object. You initialize it with an NSApplication object and the name of the event class (as declared in the ADF) to which the event belongs. You then provide a value for each of the event class fields by setting properties in the object. After you have provided values for all the event fields, you call the event collector's Write() method, passing the event object as an argument. You then repeat this process for each event you want to submit, creating a new Event object each time.

The first time you call the event collector's Write() method, it starts a new event batch. Each subsequent call adds a new event to that same batch. After you have submitted all the events that make up an event batch, you can call the event collector's Commit() method to commit all the collected events to the application database. Until you call the Commit() method on the event collector, the batch is treated as incomplete, and none of the events will be processed. After calling Commit(), the event batch will be marked complete and is eventually picked up by the generator for processing. Figure 8.6 illustrates this sequence of calls. Several sample event providers described later in this chapter use the Event Object API.

Figure 8.6. Objects and calls in the Event Object API.

[View full size image]

The EventCollector class also provides a facility for aborting the collection of an event batch. If, instead of calling Commit(), you call the Abort() method, the event batch will be abandoned in its current state, and none of the events submitted up to that point will ever be processed.

The first call to the event collector's Write() method after calling either Commit() or Abort() begins a new event batch. Thereafter, all subsequent events passed to the Write() method become part of that same batch, until either Commit() or Abort() is called again.

The XML Bulkload Event API

The XML Bulkload Event API provides a way to submit event data represented in XML form. The API takes a single XML document that contains the data for a whole batch of events and submits those events to the SQL-NS application. This saves the event provider implementation from having to manually parse the event data in the XML file.

The FileSystemWatcherProvider built-in event provider uses this API: When it detects a new XML file in the events watch directory, it passes the file to the XML Bulkload API for processing. In custom event providers, you can use this API to submit events, but you need not use the file watching mechanism that the FileSystemWatcherProvider employs. Instead, you can obtain the XML event data from a web service or another XML-based event source.

The XML Bulkload API consists of a single class, EventLoader. In event provider implementation, you create an EventLoader object and initialize it with an NSApplication object, the event provider name, the name of an event class, and the name of an XSD schema file that describes the structure of the XML files it will process. To submit an XML file containing events, you call the LoadXml() method, passing the name of the XML file. The file must contain events of the event class specified in the call to the EventLoader constructor. All the events in the file will be submitted as a single batch. Figure 8.7 illustrates this process. The XML Bulkload API is used in the sample standalone event provider developed later in this chapter (see the section, "A Dedicated Program: The XML Event Provider," p. 292).

Figure 8.7. Event submission with the XML Bulkload API.

[View full size image]

SQL Stored Procedures for Event Submission

SQL-NS provides a set of stored procedures that you can use to submit events directly from SQL code. The SQL-NS compiler creates these stored procedures in the application database when it compiles the event classes declared in the ADF. For each event class, the compiler generates the following four stored procedures:

  • NSEventBeginBatch<EventClass>Starts a new event batch

  • NSEventWrite<EventClass>Submits a single event

  • NSEventFlushBatch<EventClass>Completes an event batch

  • NSEventSubmitBatch<EventClass>Executes a query to obtain event data and submits the results of the query as a batch of events

You use the first three stored procedures if you want to submit events one at a time, and you use the fourth, NSEventSubmitBatch<EventClass>, if you want to submit a whole set of events in a single call. Figure 8.8 shows these stored procedures in use.

Figure 8.8. Use of the event submission stored procedures.

[View full size image]

In the names of the actual stored procedures, <EventClass> is replaced by the event class name. For example, the stored procedure that starts a new event batch of the SongAdded event class in the music store application will be called NSEventBeginBatchSongAdded.

You can call these stored procedures from any context in which SQL code can be invoked. One common practice is to call these stored procedures in triggers defined on tables in your application. When changes occur that invoke the triggers, the trigger code can submit the appropriate events to your SQL-NS application. In the "Submitting Events with SQL Stored Procedures" section (p. 302) later in this chapter, you'll see code that invokes these stored procedures.

Note

As you saw in previous chapters, the SQL-NS compiler also creates event class views that can be used to submit event data from SQL code via direct inserts. The main reason to use the stored procedures instead of the views is that the stored procedures offer you control over how events are grouped into batches. When you insert into an event view, each event row becomes its own batch; using the view, there is no way to group two or more events into a single batch.

Note

It's important to distinguish the stored procedures described here from the built-in SQLProvider event provider described earlier. These stored procedures are just APIs that you can use to build your own custom event provider, using SQL code. In contrast, the SQLProvider is a complete event provider that happens to use SQL queries to obtain event data. You can use the SQLProvider in your application by just customizing the SQL queries that it uses.


Building Custom Hosted Event Providers

If you need an event provider that runs within the SQL-NS engine, but none of the builtin event providers are suitable, you need to build a custom hosted event provider. Custom hosted event providers plug into the framework provided by the SQL-NS engine in the same way that the built-in event providers do.

The previous section covered the APIs that any custom event provider (hosted or not) can use to submit events. This section is about building a custom hosted event provider using some of those APIs.

Why Build a Custom Hosted Event Provider?

Building a custom event provider allows you to interface with event sources that are not well suited to either of the built-in event providers. It's certainly easy to imagine such event sources: Think about getting events from a web service or from a text feed. Neither of the built-in event providers would be of much help. The need to work with custom event sources is the most common reason for building a custom event provider.

However, there is also another important reason. If you build a custom event provider, you can perform custom processing or filtering on the data read from the event source, before submitting it to the SQL-NS application. In cases where the event source may provide duplicate or erroneous events, preprocessing the data in the event provider may allow your SQL-NS application to be more efficient and avoid errors. Filtering is also important when the event source provides an overwhelming amount of event data, only a small portion of which is relevant to your application. In a custom event provider, you can weed out irrelevant data. Some custom event providers even look at the subscriptions in the SQL-NS application to determine which events to submit. For more information on prefiltering event data in a custom event provider, see the "Prefiltering Events" section (p. 422) in Chapter 12, "Performance Tuning."

The reasons just described apply to all custom event providers, whether hosted or standalone. Let's consider some specific reasons for building a hosted custom event provider:

  • Hosted event providers run within the SQL-NS engine and therefore do not require an additional process in which to execute.

  • Because they run in the SQL-NS engine, hosted event providers run within a well-established security context; you do not have to invent a new security model just for your event provider.

  • Using a hosted event provider reduces the complexity of administering your application in deployment. For example, when you start the SQL-NS engine, the event provider starts with it. Your operations staff do not need to perform additional steps to start and stop the event provider, and they can monitor and administer it using the standard mechanisms for components in the SQL-NS engine.

Although these advantages are compelling, you shouldn't get the impression that hosted event providers are necessarily a better choice than standalone event providers. There are certain scenarios to which standalone event providers are better suited. These are discussed in the "Building Standalone Event Providers" section (p. 291).

Choosing an Event Provider Type

The "Event Providers and Event Sources" section (p. 234) describes two kinds of event sources: active and passive. Active event sources push event data out to listeners; passive event sources make data available and wait for interested parties to ask for it.

There are also two types of hosted event providers: continuous and scheduled. Continuous event providers are started and then run until they are stopped; scheduled event providers are invoked periodically, on a configurable schedule. The FileSystemWatcherProvider is an example of a continuous event provider. It starts and then continuously monitors a directory for new event files. The SQLProvider is an example of a scheduled event provider. It is invoked on a periodic schedule to execute the events query.

Although it isn't a firm rule, in general, continuous event providers are well suited to active event sources, and scheduled event providers are well suited to passive event sources. A continuous event provider typically establishes itself as a listener with the active event source and then waits to receive events. In contrast, a scheduled event provider periodically polls the event source, checking whether new event data is available.

The type of event provider type you should build depends on the particular needs of your application. This section shows examples of both continuous and scheduled event providers for the music store application. However, only the scheduled event provider really makes sense because the music store application's event source (the music store database) is passive. To test the continuous event provider, we will simulate an active event source. Admittedly, this is somewhat of an artificial scenario, but, for the purpose of explanation, it is sufficient to illustrate continuous event provider operation.

Classes, Interfaces, and Assemblies

Implementing a custom hosted event provider involves building a class that implements a SQL-NS interface. The code for the event provider class can be written in any of the .NET languages, although the examples in this book are in C#. This book assumes familiarity with object-oriented programming, but this section provides a brief recap of classes and interfaces.

Think of a class as a definition of a type of object. The class defines the data that the object encapsulates and the implementation of the methods and properties it exposes.

An interface is a description of a set of methods and properties. An interface does not provide the implementation of the methods and properties, but declares their signatures (names, arguments, and return value types).

A class can implement an interface by implementing the methods and properties that the interface describes. Many different classes can implement the same interface. You can think of an interface as a contract between a class and its callers. If a class implements a particular interface, callers know that it provides the methods and properties that the interface describes.

The interface concept is the basis for all custom components in the SQL-NS engine. For each type of custom component the SQL-NS engine supports, it defines an interface. The engine uses the methods described by those interfaces to interact with the components. Any class that implements the interface for a particular component type can plug into the SQL-NS engine as a custom component. The engine just needs to know how to load and instantiate the class, and then it can interact with it through the methods described in the interface.

In the case of hosted event providers, SQL-NS defines two interfaces: one for continuous event providers and one for scheduled event providers. These interfaces are presented in the next section. These interfaces describe the methods that the engine calls to interact with event providers. To build a custom event provider, you need to implement these methods in a class of your own.

After you write the code for your event provider class, you build it using the .NET compiler for the language you're using. This produces an assembly (a .dll file) that contains the compiled, binary representation of the class. To use the event provider in a SQL-NS application, you declare it in a <HostedProvider> element in the ADF, much like you did for the built-in event providers. In the <HostedProvider> declaration, you specify the name of the class that implements the event provider interface and the name of the assembly that contains the compiled class. At runtime, the SQL-NS engine loads the assembly and creates an object of the event provider class. In the <HostedProvider> declaration, you can also supply a set of arguments that the SQL-NS engine passes to the event provider on startup.

Hosted Event Provider Interfaces

SQL-NS defines two interfaces for hosted event providers:

  • IEventProvider for continuous event providers

  • IScheduledEventProvider for scheduled event providers

Both interfaces are defined in the Microsoft.SqlServer.NotificationServices namespace and are compiled into the SQL-NS assembly. To use these interfaces in a Visual Studio project, you need to add a reference to the SQL-NS assembly. This section describes these interfaces and the methods they define. The subsequent sections show examples of classes that implement them.

The IEventProvider Interface

Listing 8.7 shows the definition of the IEventProvider interface. The interface defines three methods, Initialize(), Run(), and Terminate(). All continuous event provider classes must implement these methods.

Listing 8.7. The IEventProvider Interface

interface IEventProvider
{
    void Initialize(
        NSApplication application,
        string providerName,
        StringDictionary args,
        StopHandler stopDelegate);

    bool Run();

    void Terminate();
}

To start an event provider, the SQL-NS engine creates an instance of the event provider class and then calls the Initialize() method. The engine passes four arguments to the Initialize() method:

  • An NSApplication object representing a connection to the SQL-NS application to which the event provider submits events

  • An event provider name, as configured in the <ProviderName> element of the provider's declaration in the ADF

  • A dictionary of arguments as specified in the <Arguments> element of the event provider's declaration in the ADF

  • A delegate that the event provider can invoke to tell the engine that it needs to stop

Tip

The Initialize() method on IEventProvider uses a StringDictionary object to pass arguments to the event provider. StringDictionary is a class in the .NET Frameworks, defined in the System.Collections.Specialized namespace. It is used in several of the SQL-NS interfaces, including IEventProvider. Whenever you write the code for a class that implements any of these interfaces, you need to put a using declaration for System.Collections.Specialized at the top of your source file.

The implementation of the Initialize() method typically stores the NSApplication object, provider name, and stop delegate in private member variables for use later. It also usually validates and stores the arguments passed in from the ADF. SQL-NS requires that the Initialize() method of every event provider implementation must return within 5 minutes or else the engine will consider it to have timed out. If this occurs, it will log an error and immediately call the provider's Terminate() method.

The SQL-NS engine calls the Run() method to tell the event provider to begin processing. In this method, the continuous event provider typically registers itself as a listener with the event source and then waits to receive events. The Run() method must also return within 5 minutes, so continuous event providers will usually create a new thread to continue processing after the Run() method returns. Run() must return true if the event provider is running successfully, false if it encountered an error and cannot run.

The Terminate() method is called by the SQL-NS engine to tell the event provider to stop. This typically occurs at the time the SQL-NS engine is shut down but can also occur if an administrator disables the event provider. Terminate() has a 1-minute window in which to return; otherwise, the event provider is terminated unconditionally. Within that 1-minute window, the event provider must stop submitting events and clean up any resources it has in use.

All the event provider methods are expected to catch and handle exceptions internally. They should not throw any exceptions out to the caller (the SQL-NS engine), unless the exception indicates a fatal error. If any event provider method does throw an exception, the SQL-NS engine logs an error and terminates the event provider immediately.

The IScheduledEventProvider Interface

Listing 8.8 shows the definition of the IScheduledEventProvider interface that scheduled hosted event providers need to implement. Notice that the methods and their signatures are the same as those defined in the IEventProvider interface.

Listing 8.8. The IScheduledEventProvider Interface

public interface IScheduledEventProvider
{
    public void Initialize(
        NSApplication application,
        string providerName,
        StringDictionary args,
        StopHandler stopDelegate);

    public bool Run();

    public void Terminate();
}

Although the methods and their signatures are the same, the way in which the Run() method is called differs from IEventProvider. In a class that implements IEventProvider, the Run() method is called only once, at the time that the event provider starts. In a class that implements IScheduledEventProvider, the Run() method is called repeatedly, according to the schedule defined in the event provider's ADF declaration. Each time the Run() method is called, the event provider can do the processing it needs to obtain and submit events, as long as it returns within a 5-minute time limit. As is the case in IEventProvider(), the Run() method must return a Boolean value indicating whether the event provider is running successfully.

The calling pattern for the Run() method is the only difference between IScheduledEventProvider and IEventProvider. The Initialize() and Terminate() methods have the same semantics across the two interfaces.

Building a Continuous Custom Hosted Event Provider

To build a continuous custom hosted event provider for the music store application, we need to create a class that implements IEventProvider. As mentioned earlier, a continuous hosted event provider doesn't make much sense in the context of the music store application, but this example is included here to illustrate the concepts behind continuous event providers in general. As you read this section, focus on the general principles, not on the specific example.

The continuous event provider we're going to build is based on Message Queuing. The "Installing Message Queuing" section (p. 245) earlier in this chapter provides instructions on how to install Message Queuing on your system. The sample described in this section will not work if you do not have Message Queuing installed.

A Brief Introduction to Message Queuing

Message Queuing is a technology that facilitates reliable transactional communication between applications. An application can establish one or more message queues on which it can exchange messages with other applications. The underlying message queuing framework provides the applications with reliability, transactional, and ordering guarantees about the queuing and retrieval of messages.

In typical message queuing scenarios, one application creates a message queue and assigns it a name. Other applications then reference this message queue by that name to send and receive messages on it. Message Queuing can be used to communicate between different parts of one application in a single process, between applications on different processes, and even between applications distributed across different machines.

The .NET Framework offers a number of Message Queuing APIs in the System.Messaging namespace. Using these APIs, applications can exchange objects as messages on queues. The sample event provider in this chapter uses these APIs.

A Message Queuing Event Provider

The event provider we build in this section establishes a message queue and then listens for messages on it. Messages sent to the queue contain event data that the event provider reads and then submits to the music store application.

We'll use a simple client program to test the event provider. This program writes some hard-coded event data to the queue, which the event provider then receives.

Message queues are identified by name; therefore, the event provider needs to assign a name to the queue it establishes. Rather than use a fixed name, hard-coded in the implementation, the event provider allows you to specify the queue name via an argument in the ADF (in the <Arguments> element of its <HostedProvider> declaration). This allows the event provider to be easily reused with different queue names. You could even declare two instances of the event provider in the ADF, each using a different queue name.

The Continuous Event Provider Class

Use the following instructions to open the Visual Studio solution containing the code for the continuous event provider:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\MQHostedEventProvider directory.
2. | Open the solution file, MQHostedEventProvider.sln, in Visual Studio.

From within the solution, open MQHostedEventProvider.cs and browse through the event provider code. Near the beginning of this file, you'll see a class called SongAddedEvent. Ignore this for now and focus on the MQHostedEventProvider class, which contains the actual event provider implementation. Listing 8.9 shows an outline of this class.

Listing 8.9. Outline of the MQHostedEventProvider Class

namespace SongAlertsCustomComponents
{public class MQHostedEventProvider : IEventProvider
    {
        public MQHostedEventProvider()
        {
            // Do nothing.
        }

        public void Initialize(
            NSApplication application,
            string providerName,
            StringDictionary args,
            StopHandler stopDelegate)
        {
            ...
        }

        public bool Run()
        {
            ...
        }

        public void Terminate()
        {
            ...
        }

        private void DoEventSubmission()
        {
            ...
        }
    }
}

Notice that the class declaration (the first line) indicates that the class implements IEventProvider. The body of the class contains the implementations of the three IEventProvider methods, Initialize(), Run(), and Terminate(). There is also a helper method (DoEventSubmission()) and a constructor.

The constructor takes no arguments and does nothing. All the initialization happens in the Initialize() method. All classes that implement SQL-NS plug-in components must have a public, parameterless constructor. This is required because the class will be instantiated by the SQL-NS engine at runtime, using the information specified in the ADF. If the constructor took parameters, the engine would not be able to instantiate the class because it would not know what values to pass for those parameters.

Let's look at the event provider class's methods, starting with Initialize() in Listing 8.10.

Listing 8.10. Implementation of the Initialize() Method in MQHostedEventProvider

public class MQHostedEventProvider : IEventProvider
{
    ...
    private NSApplication   songAlertsApp;
    private string          providerName;
    private StopHandler     stopHandler;
    private string          queueName;
    private MessageQueue    queue;
    private EventCollector  eventCollector;
    private bool            terminating;
    private Thread          submissionThread;

    ...
    public void Initialize(
        NSApplication application,
        string providerName,
        StringDictionary args,
        StopHandler stopDelegate)
    {
        // Initialize the private members.
        this.songAlertsApp = application;
        this.providerName = providerName;
        this.stopHandler = stopDelegate;

        // Get the queue name from the arguments passed in.
        if (args.ContainsKey("QueueName"))
        {
            this.queueName = args["QueueName"];
        }
        else
        {
            throw new ArgumentException(
                "A required event provider argument was not specified.",
                "QueueName");
        }

        // Initialize the message queue object.
        if (MessageQueue.Exists(queueName))
        {
            // The message queue exists, so just obtain a reference
            // to it.
            this.queue = new MessageQueue(queueName);
        }
        else
        {
            // The message queue does not exist, so create it.
            this.queue = MessageQueue.Create(queueName);
        }

        // Set the formatter to indicate body contains
        // a SongAddedEvent.
        this.queue.Formatter = new XmlMessageFormatter(
            new Type[] {typeof(SongAddedEvent)});

        // Initialize the event collector used to submit events.
        this.eventCollector = new EventCollector(
            this.songAlertsApp,
            this.providerName);

        // Initialize the terminating flag.
        this.terminating = false;

        // Initialize the thread object to null - we'll create
        // it in the Run() method.
        this.submissionThread = null;
    }
    ...
}

Initialize() begins by storing the supplied parameters (passed by the SQL-NS engine) in private member variables so that they can be used later. One of the items passed in is a dictionary of event provider arguments, constructed from the <Arguments> element in the event provider's ADF declaration. The dictionary contains one entry for each argument name and value provided in the ADF. Initialize() looks in the dictionary for an argument called QueueName. If it doesn't find this argument, it throws an exception indicating that a required argument is missing. The SQL-NS engine will catch this exception and stop the event provider immediately. If the queue name argument is found, it is stored in a private member variable.

The code then initializes a MessageQueue object that will be used later to read from the queue. It checks whether a queue with the given queue name already exists and, if not, creates it. If the queue does exist, the code just creates a new MessageQueue object that refers to it.

The next line sets the Formatter property on the queue to indicate what type of messages will be sent and received on the queue. Because the queue is used to exchange events, the messages must represent event data. Recall that at the top of the MQHostedEventProvider.cs file there is the definition of a class called SongAddedEvent. This simple class provides an encapsulation of the data associated with a SongAdded event. The value set for the Formatter property indicates that the messages sent and received on the queue will be XML-serialized instances of the SongAddedEvent class.

The event provider uses the Event Object API, so after setting the Formatter property on the queue, Initialize() creates an EventCollector object that will be used later to submit events. It passes the NSApplication object and provider name that it obtained from the SQL-NS engine to the EventCollector constructor.

The final two statements of Initialize() provide initial values for two private member variables, terminating and submissionThread. The terminating variable is a Boolean flag used to signal that the event provider is shutting down. The submissionThread variable will hold a thread object that represents the running thread used to submit events. The use of both these variables will become clear when we look at the Run() and Terminate() methods.

Listing 8.11 shows the implementation of the event provider's Run() method.

Listing 8.11. Implementation of the Run() Method in MQHostedEventProvider

public class MQHostedEventProvider : IEventProvider
{
    ...

    public bool Run()
    {
        // Create a thread to do the event submission.
        this.submissionThread =
            new Thread(new ThreadStart(this.DoEventSubmission));

        // Start the thread.
        this.submissionThread.Start();

        return true;
    }
    ...
}

The Run() method does not do any event submission itself. Rather, it creates a new thread that will do the event submission later. The new thread is needed because Run() must return within a 5-minute time window. The thread that calls Run() belongs to the SQL-NS engine, and the event provider cannot use it to listen for messages on the queue indefinitely. Run() just establishes a thread that will do the listening and event submission, and then returns.

To create the thread, Run() instantiates a new Thread object and passes it a delegate to the DoEventSubmission() helper method. When the thread starts up, it begins executing in this method. After creating the thread object, Run() calls its Start() method and returns TRue, indicating success.

Before looking at the DoEventSubmission() method, which does most of the work of the event provider, let's look at the final IEventProvider method, Terminate(). Listing 8.12 shows the code for Terminate().

Listing 8.12. Implementation of the Terminate() Method in MQHostedEventProvider

public class MQHostedEventProvider : IEventProvider
{
    ...

    public void Terminate()
    {
        // Set the terminating flag.
        lock(this)
        {
            this.terminating = true;
        }

        // Now wait for the submission thread to exit.
        this.submissionThread.Join();
    }
    ...
}

The Terminate() method sets the terminating flag to indicate to the event submission thread that the event provider needs to shut down (as you'll see later, the submission thread checks this flag periodically). Because the terminating flag can be accessed by multiple threads (the thread that calls Terminate sets it and the event submission thread reads it), the lock construct is used to synchronize access to it. After setting the terminating flag, Terminate() then calls the Join() method on the thread object, which blocks until the thread completes.

Let's examine DoEventSubmission(), the method that does the actual event submission. The event submission thread created by Run() begins in this method and stays in this method until it exits. Note that DoEventSubmission() is not a method defined by IEventProvider but rather a helper method implemented by the MQHostedEventProvider class. Listing 8.13 shows the code for DoEventSubmission().

Listing 8.13. Implementation of the DoEventSubmission() Method in MQHostedEventProvider

public class MQHostedEventProvider : IEventProvider
{
    private const string    EVENT_CLASS_NAME = "SongAdded";
    ...
    private void DoEventSubmission()
    {
        TimeSpan    timeout = new TimeSpan(0, 0, 5); // 5 seconds
        bool        shouldTerminate;

        lock(this)
        {
            shouldTerminate = this.terminating;
        }

        while (!shouldTerminate)
        {

            try
            {
                // Read a message from the queue. Specify a
                // timeout value so that we don't block forever
                // and can check the terminating flag.
                Message m = queue.Receive(timeout);

                // Extract the SongAddedEvent from the body.
                SongAddedEvent e = (SongAddedEvent)m.Body;

                // Create a SQL-NS event and submit it.
                Event evt =
                    new Event(songAlertsApp, EVENT_CLASS_NAME);
                evt["SongId"] = e.SongId;

                eventCollector.Write(evt);
                eventCollector.Commit();
            }
            catch (MessageQueueException mex)
            {
                // If the exception indicated that a timeout
                // occurred, then we absorb it and just let
                // the loop run again. Any other error means
                // something went wrong and we call the stop
                // delegate.
                if (mex.MessageQueueErrorCode !=
                      MessageQueueErrorCode.IOTimeout)
                {
                    this.stopHandler();
                    break;
                }
            }
            catch (Exception)
            {
                // For all other exceptions, call the stop handler.
                this.stopHandler();
                break;
            }

            lock(this)
            {
                shouldTerminate = this.terminating;
            }
        }
    }
    ...
}

The method begins by declaring a TimeSpan object that represents a duration of 5 seconds. This will be used to specify a timeout value when reading from the queue. Before doing any actual work, DoEventSubmission() checks the terminating flag (inside a lock construct for thread safety) and stores its current value in a local variable that can be checked without locking.

It then enters a while loop. In this loop, it reads from the message queue, submits an event when a message arrives, and checks the terminating flag again. The loop continues until either the terminating flag is set or a fatal error occurs.

To read from the queue, DoEventSubmission() calls the Receive() method on the queue object, passing it the 5-second timeout value. This call blocks until either a message is received or 5 seconds pass and a timeout occurs.

When a message is received, its contents are stored in a Message object. This message object contains a serialized copy of a SongAddedEvent object. The code obtains the SongAddedEvent object by casting the Body method of the Message object. It then creates a new Event object (recall that Event is a class in the Event Object API) and initializes it with the NSApplication object representing the application and the event class name. It then sets the SongId field in the event object to the value stored in the SongAddedEvent object just read from the queue.

After the event object is properly initialized, the code passes it to the EventCollector's Write() method. It then immediately calls Commit() on the event collector to close the event batch. Thus, this event provider produces event batches containing single events.

If a timeout occurs while reading from the message queue, the Receive() method throws a MessageQueueException object, with an error code value of IOTimeout. The code that reads from the message queue appears within a try block and catches exceptions of type MessageQueueException. The code only treats a MessageQueueException as a real error condition if the error code is anything other than IOTimeout. In the case of exceptions generated for timeouts, the exceptions are just ignored, and the loop continues.

The timeout is needed to facilitate periodic checking of the terminating flag. If DoEventSubmission() just did a blocking read on the queue, without checking this flag, a request to terminate the event provider might go ignored until a message is actually received. This would obviously cause problems because the Terminate() method waits for the submission thread to complete, and this must happen within a 1-minute window. In general, continuous event providers must be written in such a way that they are responsive to terminate requests.

Building the Continuous Event Provider Assembly

Build the MQHostedEventProvider project in Visual Studio. This compiles the source code we just looked at and produces the event provider assembly.

You will find the event provider assembly in the bin\Debug output directory under the MQHostedEventProvider project directory. Navigate to C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\MQHostedEventProvider\bin\Debug and verify that MQHostedEventProvider.dll has been created.

Declaring the Continuous Custom Hosted Event Provider in the ADF

To use the continuous custom hosted event provider, we need to declare it in the ADF in a <HostedProvider> element. Listing 8.14 shows this declaration.

Listing 8.14. Declaration of the Continuous Custom Hosted Event Provider in the ADF

[View full width]
<Application>
  ...
  <Providers>
    ...
    <HostedProvider>
      <ProviderName>SongAddedMQProvider</ProviderName>
      <ClassName>SongAlertsCustomComponents.MQHostedEventProvider</ClassName>
      <AssemblyName>%_ApplicationBaseDirectoryPath_%\CustomComponents
  \MQHostedEventProvider\bin\Debug\MQHostedEventProvider.dll</AssemblyName>
      <SystemName>%_NSServer_%</SystemName>
      <Arguments>
        <Argument>
          <Name>QueueName</Name>
          <Value>.\Private$\SongAddedEventQueue</Value>
        </Argument>
      </Arguments>
    </HostedProvider>

    ...
  </Providers>

  ...
</Application>

As with other <HostedProvider> declarations you've seen, the <ProviderName> element assigns a name to the event provider. The <ClassName> element specifies the name of the event provider class. As shown in Listing 8.9, the event provider class name is MQHostedEventProvider and it is declared in the namespace, SongAlertsCustomComponents. Thus, the fully qualified name of the event provider class is SongAlertsCustomComponents.MQHostedEventProvider. This fully qualified name is given as the value for the <ClassName> element in Listing 8.14.

Caution

Whenever you refer to a class that implements a SQL-NS custom component in the ADF, you must specify its fully qualified name, including any namespace prefix.

The <AssemblyName> element provides the full path to the event provider assembly. In the example shown in Listing 8.14, this path is derived from the _ApplicationBaseDirectoryPath_ parameter and points to the event provider assembly you just built. Note that even though we declare both custom and built-in hosted event providers with the same <HostedProvider> element, the <AssemblyName> element is required only in the case of custom event providers.

As before, the <SystemName> element specifies the machine on which the event provider should run, and the <Arguments> element specifies the arguments that get passed to the event provider at startup (via the args parameter to the Initialize() method). Our event provider requires a single argument, QueueName, that specifies the name of the event queue that will be used to receive events.

Testing the Continuous Custom Hosted Event Provider

To test the custom hosted event provider, add the code shown in Listing 8.14 to the ADF. You can find this code in the supplemental ADF, ApplicationDefinition-9.xml. After you've added the code to the ADF, update your instance using the update instructions provided earlier in this chapter.

Caution

Before updating your instance, make sure that you have built the event provider assembly successfully, as described in the "Building the Continuous Event Provider Assembly" section (p. 278). When you start the SQL-NS service after the update, it will try to load the custom event provider from this assembly. If it can't find the assembly, the service will log an error to the Application Event Log. The message in the log will have the ID 2008 and its description will read:

  Notification Services failed to load the event provider assembly.

When you start the SQL-NS service after updating your instance, the new custom hosted event provider starts. We now need to send a message to the message queue to see this event provider working. I've provided a simple test program that connects to the message queue and writes a single hard-coded event. Perform the following instructions to build and run this program:

1.
Navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\Test\MQClient directory.

2.
Open the solution file MQClient.sln in Visual Studio.

3.
Build the MQClient project.

4.
In a command prompt, navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\Test\MQClient\bin\Debug directory.

5.
Run the MQClient.exe program. This should produce the following output:

Connected to message queue: .\Private$\SongAddedEventQueue
Wrote SongAdded event with SongId=15

As the output indicates, the client program writes a single SongAdded event to the message queue, with the SongId field set to 15. Verify that the event provider submitted this event to the SQL-NS application by examining the contents of the events table in Management Studio as you did before.

Querying the Nseventbatchview

Our application now has three event providers: the FileSystemWatcherProvider, the SQLProvider, and the custom hosted event provider we just added. Looking at the events and event batches tables, it's not always clear which event provider submitted a particular event batch. To find out, you can query the NSEventBatchView that the SQL-NS compiler creates in every application schema. This view returns a row for every event batch submitted into the application. Columns in the view indicate the event class that the events in the batch belong to and the name of the event provider that submitted them.

To query the view in the music store application, you can issue the following commands in Management Studio:

  USE MusicStore
  SELECT * FROM [SongAlerts].[NSEventBatchView]

The rows returned by this query should list event batches from all three of the application's event providers. The provider names we configured in the ADF were SongAddedFSWProvider for the FileSystemWatcher, SongAddedSQLProvider for the SQLProvider, and SongAddedMQProvider for our continuous custom hosted event provider. You should see these provider names in the ProviderName column in the resultset. Event batches submitted by directly inserting rows into the events view (as we did in previous chapters) have NULL in the ProviderName column.

Building a Scheduled Custom Hosted Event Provider

In this section, we build a scheduled custom hosted event provider. Both the development process and the code for this event provider are similar to what you just saw in the previous section for the continuous hosted event provider.

To build the scheduled event provider, we create a class that implements IScheduledEventProvider. We then configure this event provider in the ADF in a <HostedProvider> element, which will specify an invocation schedule.

Operation of the Scheduled Event Provider

The scheduled event provider described in this section polls the music store database to find new songs that have been added. It remembers the last time it ran and looks for songs with timestamps between that last time and the current time. It then submits SongAdded events for the song IDs it finds.

This is similar to the way the SQLProvider works, but rather than running the events query and doing event submission in the database, this event provider reads the data from the event source into memory. When the data is in memory, it is then submitted using the Event Object API. Building an event provider this way provides an opportunity to process or filter the data from the event source in memory before submitting it as events to the SQL-NS application. However, this requires a trade-off with respect to efficiency, as described in the section "The SQLProvider" (p. 253). Transferring the data in and out of the database repeatedly can be inefficient.

The Scheduled Event Provider Class

Use the following instructions to open the Visual Studio solution containing the code for the scheduled event provider:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\ScheduledHostedEventProvider directory.
2. | Open the solution file ScheduledHostedEventProvider.sln in Visual Studio.

Within the solution, open ScheduledHostedEventProvider.cs and browse through the event provider code. The event provider class is called ScheduledHostedEventProvider and implements IScheduledEventProvider. Listing 8.15 shows the class declaration and the implementation of the Initialize() method of IScheduledEventProvider.

Listing 8.15. Implementation of the Initialize() Method in ScheduledHostedEventProvider

namespace SongAlertsCustomComponents
{
    ...
    class ScheduledHostedEventProvider : IScheduledEventProvider
    {
        ...
        private const string    DATABASE_NAME = "MusicStore";
        private NSApplication   songAlertsApp;
        private string          providerName;
        private StopHandler     stopHandler;

        private EventCollector  eventCollector;

        private string          serverName;
        private string          sqlUserName;
        private string          sqlPassword;

        private string          connectionString;
        private SqlCommand      command;
        private DateTime        lastRunTime;

        public void Initialize(
            NSApplication application,
            string providerName,
            StringDictionary args,
            StopHandler stopDelegate)
        {
            // Initialize the private members.
            this.songAlertsApp = application;
            this.providerName = providerName;
            this.stopHandler = stopDelegate;

            // Initialize the event collector used to submit events.
            this.eventCollector = new EventCollector(
                this.songAlertsApp,
                this.providerName);

            // Read the arguments passed in.
            if (args.ContainsKey("DatabaseServerName"))
            {
                this.serverName = args["DatabaseServerName"];
            }
            else
            {
                throw new ArgumentException(
                    "A required event provider argument was not specified.",
                    "DatabaseServerName");
            }

            AuthenticationMode authMode
                = AuthenticationMode.WindowsAuthentication;
                // Treat a blank username as if no username was specified.
                if (args.ContainsKey("SqlUserName") &&
                    args["SqlUserName"] != "")
                {
                    this.sqlUserName = args["SqlUserName"];
                    authMode = AuthenticationMode.SqlServerAuthentication;
                }
                else
                {
                    // This argument is optional, so no need to throw
                    // an error if it wasn't supplied.
                    this.sqlUserName = null;
                }

                if (args.ContainsKey("SqlPassword")
                    && args["SqlPassword"] != "")
                {
                    this.sqlPassword = args["SqlPassword"];
                }
                else
                {
                    // This argument is optional, so no need to throw
                    // an error if it wasn't supplied.
                    this.sqlPassword = null;
                }

                // The SQL username and password must either be
                // supplied together, or neither of them must be
                // supplied. It's invalid to supply just one or
                // the other.
                if (this.sqlUserName == null ^ this.sqlPassword == null)
                {
                    string message;
                    string argument;

                    if (this.sqlUserName == null)
                    {
                        message = "A SQL password was supplied without a username";
                        argument = "SqlPassword";
                    }
                    else
                    {
                        message = "A SQL username was supplied without a password";
                        argument = "SqlUserName";
                    }

                    throw new ArgumentException(message, argument);
                }

                // Build the database connection string.
                this.connectionString = BuildConnectionString(
                    this.serverName,
                    DATABASE_NAME,
                    true,
                    authMode,
                    this.sqlUserName,
                    this.sqlPassword,
                    this.providerName);

                // Create the SQL command object that will read new songs
                // from the database.
                this.command = new SqlCommand();

                this.command.CommandText = @"
    SELECT SongId FROM [Catalog].[SongDetails]
    WHERE DateAdded > @LastRunTime AND DateAdded <= @CurrentRunTime";

                this.command.CommandType = CommandType.Text;

                this.command.Parameters.Add(
                    "@LastRunTime",
                    SqlDbType.DateTime);

                this.command.Parameters.Add(
                    "@CurrentRunTime",
                    SqlDbType.DateTime);

                // Initialize the last run time.
                this.lastRunTime = DateTime.UtcNow;
        }
        ...
    }
}

The Initialize() method stores the arguments it receives from the SQL-NS engine in private member variables, creates an event collector for use later, and then validates the ADF arguments. Because this event provider needs to connect to the music store database, the ADF arguments must provide the database server name and a SQL username and password if SQL Authentication is used. Initialize() obtains values for these arguments from the dictionary passed in and uses them to build a connection string to the database.

Initialize() then creates a SqlCommand object that will be used to query the music store database for new songs. This command selects the song IDs from the music store catalog's SongDetails view, where the song timestamp falls between the last and current runtime of the event provider. The command text is specified using parameters for the last runtime and current runtime. Later, when the event provider executes the command, it provides values for these parameters.

Finally, Initialize() stores the current UTC time in the private member variable, lastRunTime. This variable is used to track the last time the event provider was run. Initializing this variable with the current UTC time on startup means that on the first run, the event provider looks for songs added since its start time.

Listing 8.16 shows the implementation of the Run() and Terminate() methods on the scheduled event provider class.

Listing 8.16. Implementation of the Run() and Terminate() Methods in ScheduledHostedEventProvider

namespace SongAlertsCustomComponents
{
    ...
    class ScheduledHostedEventProvider : IScheduledEventProvider
    {
        private const string    EVENT_CLASS_NAME = "SongAdded";
        ...
        public bool Run()
        {
            // Create a database connection.
            SqlConnection connection =
                new SqlConnection(this.connectionString);
            int eventsSubmitted = 0;
            bool success = false;

            // Get the current time.
            DateTime currentRunTime = DateTime.UtcNow;

            try
            {
                // Open the connection.
                connection.Open();
                using (connection)
              {
                  // Set the command's Connection property to the
                  // current connection.
                  this.command.Connection = connection;

                  // Set the command parameter values.
                  command.Parameters["@LastRunTime"].Value =
                      this.lastRunTime;
                  command.Parameters["@CurrentRunTime"].Value =
                      currentRunTime;

                  // Execute the command.
                  SqlDataReader reader = command.ExecuteReader();

                  using (reader)
                  {
                      // Read rows from the resultset and
                      // submit the Song Ids as events.
                      while (reader.Read())
                      {
                          int songId = (int) reader["SongId"];

                          Event evt = new Event(
                              songAlertsApp,
                              EVENT_CLASS_NAME);

                          evt["SongId"] = songId;

                          eventCollector.Write(evt);
                          eventsSubmitted++;
                      }
                  }

                  success = true;
              }
        }
        catch (Exception)
        {
            // Don't call the stop delegate here because
            // one error should not cause the event provider
            // to shut down. Next time Run() is called, we'll
            // just try again.
            // Set the success flag to false so we know to roll
            // back the event batch.
            success = false;
        }
        finally
        {
            // If no events were submitted, then there's no
            // work to do here.
            if (eventsSubmitted > 0)
            {
                // If the submission was successful, commit
                // the event batch, otherwise abort it.
                if (success)
                {
                    this.eventCollector.Commit();

                    // Only set the last run time to the current
                    // time now that we've successfully committed
                    // the event batch.
                    this.lastRunTime = currentRunTime;
                }
                else
                {
                    this.eventCollector.Abort();
                }
             }
          }

          return true;
        }

        public void Terminate()
        {
            // Nothing to do.
        }
        ...
    }
}

Recall that the key difference between the Run() method on a scheduled event provider and the Run() method on a continuous event provider is the frequency at which it is invoked. A continuous event provider's Run() method is invoked once and must create a separate thread to do any long-running operations. In contrast, the Run() method on the scheduled event provider is called repeatedly (on the schedule specified in the ADF), so it can implement the periodic detection and submission of events directly.

The Run() method shown in Listing 8.16 begins by obtaining the current time and storing this in a local variable. It then opens a connection to the database, specifies values for the last and current time parameters in the SQL command that queries the music store database, and then executes the command. For each row of data returned by the query, Run() creates an event object, sets the value of the SongId field, and then submits it to the event collector.

The finally block, which executes before Run() returns, commits the event batch if everything went successfully or aborts it if a failure occurred. In the case of success, the lastRunTime private member variable is updated with the current time.

Caution

Whenever you use the Event Object API, you must write your error handling code so that any open event batches are either committed or rolled back if an error occurs. If you abandon an event batch without either committing or aborting it, the data associated with that event batch will never be processed or cleaned up properly.

Notice that the Run() method always returns true. This indicates to the SQL-NS engine that it should be called again, at the next time specified by the schedule. If Run() returns false, the SQL-NS engine terminates the event provider immediately. In the case of this event provider, Run() returns true even if a failure occurs, so that it can make another attempt at checking for events the next time it is called. In other scheduled event providers, a single failure may warrant termination, in which case Run() should return false.

The Terminate() method shown in Listing 8.16 does nothing. The scheduled event provider does not keep any long-lived state that it needs to clean up on shutdown, so there is no work for Terminate() to do. This is different from the Terminate() method on the continuous event provider, which had to coordinate the shutdown of the worker thread.

Building the Scheduled Event Provider Assembly

Build the ScheduledHostedEventProvider project in Visual Studio. This compiles the source files and produces the event provider assembly.

After building the project, you will find the event provider assembly in the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\ScheduledHostedEventProvider\bin\Debug directory. The assembly is called ScheduledHostedEventProvider.dll.

Declaring a Scheduled Custom Hosted Event Provider in the ADF

Listing 8.17 shows the <HostedProvider> declaration for the scheduled custom hosted event provider. This is similar to the declaration you saw for the continuous custom hosted event provider, except that it specifies a schedule.

Listing 8.17. Declaration of the Scheduled Custom Hosted Event Provider in the ADF

[View full width]
<Application>
  ...
  <Providers>
    ...
    <HostedProvider>
      <ProviderName>SongAddedScheduledProvider</ProviderName>
      <ClassName>SongAlertsCustomComponents.ScheduledHostedEventProvider</ClassName>
      <AssemblyName>%_ApplicationBaseDirectoryPath_%\CustomComponents
  \ScheduledHostedEventProvider\bin\Debug\ScheduledHostedEventProvider.dll</AssemblyName>
      <SystemName>%_NSServer_%</SystemName>
      <Schedule>
        <Interval>P0DT00H00M10S</Interval>
      </Schedule>
      <Arguments>
        <Argument>
          <Name>DatabaseServerName</Name>
          <Value>%_SQLServer_%</Value>
        </Argument>
        <Argument>
          <Name>SqlUserName</Name>
          <Value></Value>
        </Argument>
        <Argument>
          <Name>SqlPassword</Name>
          <Value></Value>
        </Argument>
      </Arguments>
    </HostedProvider>
    ...
  </Providers>
  ...
</Application>

The <ClassName> element specifies the name of the event provider class, qualified with its namespace. The <AssemblyName> element specifies the full path to the event provider assembly.

Because this is a scheduled event provider, the declaration includes a <Schedule> element. This is the same type of <Schedule> element that we used to configure the SQLProvider earlier in this chapter. It follows the syntax of the XML duration data type, which is explained in the sidebar "The XSD duration Data Type" (p. 157) in Chapter 5. The value given in Listing 8.17 specifies a schedule interval of 10 seconds.

The <Arguments> section of the event provider declaration specifies the name of the database server hosting the music store database and the SQL username and password with which the event provider should connect (for SQL Server Authentication). The event provider's Initialize() method (shown earlier in Listing 8.15) processes these arguments. If the SqlUserName and SqlPassword argument values are left blank, the event provider will connect using Windows Authentication.

The original description of argument encryption in the early part of this chapter stated that it's useful when the event provider arguments contain sensitive information. This is an example of such a case: The event provider arguments may contain a username and password that should not be readily accessible to anyone looking at your application database. Because we're using argument encryption in this chapter's instance, the values of these event provider arguments will be encrypted before they are stored in the database by the compiler.

Testing the Scheduled Custom Hosted Event Provider

Add the code shown in Listing 8.17 to the ADF. You will find this code in the ApplicationDefintion-10.xml supplemental ADF. If you're using SQL Server Authentication, make sure the values of the SqlUserName and SqlPassword event provider arguments are set to the username and password of the test account you created in Chapter 2. (If you're using Windows Authentication, these arguments should be left blank.) If you configured your development environment for SQL Server Authentication in Chapter 2, the SqlUserName and SqlPassword argument values in the supplementary ADF will be preset to the username and password of your SQL test account.

After you've added the code to your ADF, update your instance according to the instructions given earlier. Before you do this, make sure that you have built the scheduled event provider code so that the event provider assembly exists. When you start the service after doing the update, the event provider will start.

Because this event provider reads song data directly from the music store database, all you need to do to test it is add a few new songs. As you did earlier, you can use the AddSongs program to accomplish this. When you use AddSongs, make sure that you leave the Submit Events for Songs Added check box unchecked.

After adding songs to the database, wait at least 10 seconds (recall that 10 seconds is the period of the scheduled event provider) and then check the events table. You should see events submitted for the songs you added. You will eventually see two events submitted for each new song because both the built-in SQLProvider we configured earlier and the custom event provider we built here will detect the new songs and submit events. The second set of events may take a little longer to appear in the events table because the SQLProvider runs on a period of 30 seconds. You can distinguish the events submitted by each event provider by querying the NSEventBatchView, as described in the sidebar, "Querying the NSEventBatchView," (p. 280), earlier in this chapter.


Building Standalone Event Providers

In this section, we add three new event providers to the application. These are standalone event providers, so they don't run in the SQL-NS engine. The examples in this section illustrate three ways that standalone event providers can be structured. In addition to the Event Object API, the examples in this section employ the event submission APIs that have not yet been used in this chapter.

Why Build a Standalone Event Provider?

Standalone event providers are not subject to the constraints imposed by the runtime environment of the SQL-NS engine, so they offer greater flexibility than hosted event providers. When designing a standalone event provider, you have the freedom to choose where it runs, how it is started, and what security and process model it follows.

Another good reason for building a standalone event provider is that you can embed it within another program. If you already have a program that acts as an event source, you can put the event provider functionality into this program instead of relying on an external event provider. For example, the AddSongs program we used to add song data to the music store application can actually be an event provider. When it adds new song records, it can also submit the corresponding events to the SQL-NS application directly. This simplifies the whole application because it removes the need for a separate event provider component (like the SQLProvider or the custom scheduled hosted event provider we built in the previous section) to read the song records and submit the corresponding event data. This section examines the code required to embed event provider functionality in the AddSongs program as well as build other types of standalone event providers.

Standalone Event Provider Configuration in the ADF

To use a standalone event provider in an application, it must be configured in the ADF. For each standalone event provider, you add a <NonHostedProvider> element to the <Providers> element. The <Providers> element may contain both hosted and standalone event provider declarations, but the <NonHostedProvider> declarations must appear after all the <HostedProvider> declarations.

Standalone event provider declarations are simple: The only element inside <NonHostedProvider> is <ProviderName>, which specifies a name for the standalone event provider. This name isn't really used by SQL-NS internally, except to validate the provider when submitting events; it mainly serves as an identification mechanism for tracking which batches the event provider submitted. SQL-NS does not need any other configuration information about standalone event providers because it does not host and invoke them in its engine.

You've already seen one example of a <NonHostedEventProvider> element: the declaration the test event provider used during initial testing of the music store prototype. Listing 8.18 shows the declaration of the three standalone event providers described in this section.

Listing 8.18. Declaration of the Standalone Event Providers in the ADF

<Application>
  <Providers>
    ...
    <NonHostedProvider>
      <ProviderName>SongAddedXMLProvider</ProviderName>
    </NonHostedProvider>
    <NonHostedProvider>
      <ProviderName>SongAddedEmbeddedProvider</ProviderName>
    </NonHostedProvider>
    <NonHostedProvider>
      <ProviderName>SongAddedSPProvider</ProviderName>
    </NonHostedProvider>
  </Providers>
  ...
</Application>

Each declaration specifies the provider name:

  • SongAddedXMLProvider is the XML-based event provider described in the section "A Dedicated Program: The XML Event Provider" (p. 292).

  • SongAddedEmbeddedEventProvider is the event provider embedded in the AddSongs program, described in the section "Submitting Events from an Application" (p. 296).

  • SongAddedSPProvider is the event provider that uses SQL stored procedures to submit events, described in the section "Submitting Events with SQL Stored Procedures" (p. 302).

Because these declarations are so simple, we'll add them all at once rather than one at a time as we look at each example. The code in Listing 8.18 can be found in the supplemental ADF, ApplicationDefinition-11.xml. Add this code to your ADF and then update your instance to compile the changes. You must do this before running any of the code in the remainder of this section.

A Dedicated Program: The XML Event Provider

In this section, we build a standalone event provider that uses the XML Bulkloader event submission API. This is an example of an event provider as a dedicated program: It's only purpose is to submit events.

This event provider can be invoked directly from the command line to manually submit events. As such, it makes a handy tool to get events into the application for testing.

Design of the XML Event Provider

The XML event provider is a command-line program that requires two arguments:

  • The name of an XML file containing event data for the music store application

  • The name of a schema file describing the contents of the XML event file

The XML event provider is invoked at the command line as follows:

XMLEventProvider -in <XmlDataFile> -schema <SchemaFile>

<XmlDataFile> is the name of the XML file containing event data, and <SchemaFile> is the name of the schema file. The XML event provider loads the XML file and submits the data in it as events, using the schema file to map the XML data to fields in the event class.

The XML event provider also takes two optional arguments, a SQL username and password to use to connect to the database for SQL Server Authentication. If you're using SQL Server Authentication, the XML event provider is invoked with the two optional arguments as follows:

[View full width]

XMLEventProvider -in <XmlDataFile> -schema <SchemaFile> -sqlusername <SqlUserName>
   -sqlpassword <SqlPassword>

<SqlUserName> is the SQL username with which to connect, and <SqlPassword> is the corresponding password.

The XML event provider uses the XML Bulkload API, which employs the same XML processing facilities as the FileSystemWatcherProvider. The XML event provider is in fact similar to the FileSystemWatcherProvider: It uses the same format for the XML event files and the schema files that describe them. The main difference is that the FileSystemWatcherProvider monitors a directory and picks up new files as they are added, whereas the XML event provider is invoked manually to process a specific file.

XML Event Provider Implementation

Use the following instructions to open the Visual Studio solution containing the code for the XML event provider:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\XMLEventProvider directory.
2. | Open the solution file, XMLEventProvider.sln, in Visual Studio.

From the Solution Explorer, open XMLEventProvider.cs, which contains the implementation of the event provider. Most of the code in this file deals with parsing and validating command-line arguments. You should take a brief look at that code to see how it works, but it isn't covered here. The Main() method, which is the entry point for the program and drives its execution, contains the code that submits events. Listing 8.19 shows the code for the Main() method.

Listing 8.19. Implementation of the Main() Method in the XML Event Provider

    class XMLEventProvider
    {
        private const string   INSTANCE_NAME = "MusicStore";
        private const string   APPLICATION_NAME = "SongAlerts";

        private const string   PROVIDER_NAME = "SongAddedXMLProvider";
        private const string   EVENT_CLASS_NAME = "SongAdded";

        static private string  xmlDataFile = null;
        static private string  schemaFile = null;

        static private string  sqlUserName = null;
        static private string  sqlPassword = null;

        [STAThread]
        static int Main(string[] args)
        {
            int retCode = 0;

            // Process the command-line arguments and display an
            // error message if appropriate.
            try
            {
                ValidateArguments(args);
            }
            catch (ArgumentException ex)
            {
                Console.WriteLine("Invalid command line arguments");
                Console.WriteLine(ex.Message);
                ShowUsage();
                retCode = 1;
            }

            // If the command-line arguments were valid,
            // we can continue.
            if (retCode == 0)
            {
                try
                {
                    // Create the SQL-NS Instance and
                    // Application objects.
                    NSInstance musicStoreInstance = null;

                    if (string.IsNullOrEmpty(sqlUserName) &&
                        string.IsNullOrEmpty(sqlPassword))
                    {
                        musicStoreInstance =
                            new NSInstance(INSTANCE_NAME);
                    }
                    else
                    {
                        musicStoreInstance = new NSInstance(
                            INSTANCE_NAME,
                            sqlUserName,
                            sqlPassword);
                    }

                    NSApplication songAlertsApp = new NSApplication(
                        musicStoreInstance,
                        APPLICATION_NAME);

                    // Create the event loader object.
                    EventLoader eventLoader = new EventLoader(
                        songAlertsApp,
                        PROVIDER_NAME,
                        EVENT_CLASS_NAME,
                        schemaFile);

                    // Load the XML file.
                    eventLoader.LoadXml(xmlDataFile);

                    Console.WriteLine("Loaded events successfully.");
                }
                catch (Exception ex)
                {
                    Console.WriteLine("An error occurred");
                    Console.WriteLine(ex.Message);
                    retCode = 1;
                }
            }

            return retCode;
        }
    ...
}

The first part of the code calls a helper method, ValidateArguments(), to parse the command-line arguments and ensure that they're valid. (You can explore the code for this function in the source file even though it isn't shown here.) If ValidateArguments() finds the required arguments, it stores their values in the xmlDataFile, schemaFile, sqlUserName, and sqlPassword member variables. If ValidateArguments() fails to find the correct arguments, it throws an exception, which Main() catches, and displays an error message.

If valid command-line arguments were obtained, the event provider constructs an NSInstance and NSApplication object to connect to the SQL-NS instance and application. Notice the two methods of constructing the NSInstance object, depending on whether Windows or SQL Server Authentication is used.

After creating the instance and application objects, the code creates a new EventLoader object. (Recall that EventLoader is a class in the XML Bulkload API.) The EventLoader constructor takes the application object, provider name (as configured in the ADF), event class name, and schema file.

Main() then calls the LoadXml() method on the EventLoader object, passing it the name of the XML data file. This method loads the XML file and submits its data as events. This is all it takes to submit events from XML files using the XML Bulkload API.

Testing the XML Event Provider

Build the XML event provider solution in Visual Studio. This produces the executable that you can invoke to submit events. In the XMLEventProvider project directory, a subdirectory called SampleFiles contains an event data file and schema file you can use to test the event provider. These files are copies of the files we used earlier with the FileSystemWatcherProvider. The following instructions describe how to perform the test:

1.
From a command prompt, navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\XMLEventProvider\bin\Debug directory.

2.
Invoke the XML event provider as follows if you're using Windows Authentication:

[View full width]

XMLEventProvider.exe -in ..\..\SampleFiles\EventData.xml -schema..\..\SampleFiles
  \SongAddedEventSchema.xsd

If you're using SQL Server Authentication, include the two optional arguments as follows:

[View full width]

XMLEventProvider.exe -in ..\..\SampleFiles\EventData.xml -schema..\..\SampleFiles
  \SongAddedEventSchema.xsd -sqlusername <SqlUserName> -sqlpassword <SqlPassword>

Replace <SqlUserName> and <SqlPassword> with the username and password of your SQL development account.

You can then check your events table to verify that the correct event data has been added.

Submitting Events from an Application

As described earlier, event provider functionality can be embedded in an existing application. In the case of the music store, the AddSongs program (which we've been using to add new song data to the music store) can act as an event provider by calling the event submission APIs to submit events for the song records it adds. This functionality has already been built in to the AddSongs program, but we have not used it so far. In this section, we'll take a look at the portion of the AddSongs code that implements the event provider operations.

Note

Note that most of the AddSongs code implements the user interface. Feel free to browse this code if you are curious about how it works, but it is not explained here because it isn't relevant to the event provider discussion.

Open the AddSongs Visual Studio solution from C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs. From the Solution Explorer, open the AddSongs.cs file. By default, this file may open in Design View, which shows the layout of the visual controls. You should open the file in Code View, so that you can see the source code in it. To do this, right-click AddSongs.cs in the Solution Explorer and select View Code.

Recall that the AddSongs user interface has a check box labeled Submit Events for Songs Added. Checking this box turns on the event provider functionality. Earlier, we left this box unchecked because we were testing other event providers and did not want AddSongs to submit events.

When the state of the check box is changed, the code shown in Listing 8.20 runs.

Listing 8.20. Event Handler for the Check Box in the AddSongs Program

public class AddSongs : System.Windows.Forms.Form
{
    ...
    private const string   INSTANCE_NAME = "MusicStore";
    private const string   APPLICATION_NAME = "SongAlerts";

    private NSInstance     musicStoreInstance;
    private NSApplication  songAlertsApp;
    ...
    private void checkBoxSubmitEvents_CheckedChanged(
        object sender,
        System.EventArgs e)
    {
        // Establish the connection to the NS instance
        // and application, if they have not already
        // been established.

        if (this.checkBoxSubmitEvents.Checked == true)
        {
            if (this.musicStoreInstance == null)
            {
                try
                {
                   if (AuthenticationMode.WindowsAuthentication ==
                       authMode)
                   {
                       musicStoreInstance =
                           new NSInstance(INSTANCE_NAME);
                   }
                   else
                   {
                       musicStoreInstance = new NSInstance(
                           INSTANCE_NAME,
                           sqlUserName,
                           sqlPassword);
                   }

                   songAlertsApp = new NSApplication(
                       musicStoreInstance,
                       APPLICATION_NAME);
                }
                catch (NSException ex)
                {
                    MessageBox.Show(
                        ex.Message,
                        "Unable to connect to Music Store instance",
                        MessageBoxButtons.OK,
                        MessageBoxIcon.Error);

                    // Set the checkbox back to unchecked.
                    this.checkBoxSubmitEvents.Checked = false;
                }
            }
        }
    }
    ...
}

The AddSongs class has private member variables, musicStoreInstance and songAlertsApp, that store instance and application objects. The first time the box is checked, the code in Listing 8.20 initializes these objects. Even if the box is unchecked after that, these objects stay initialized and will be used again if the box is checked again later.

The code that submits the events appears in the event handler for the Add to Database button. Listing 8.21 shows this code.

Listing 8.21. The Event Handler for the Add to Database Button

public class AddSongs : System.Windows.Forms.Form
{
    ...
    private const string   PROVIDER_NAME = "SongAddedEmbeddedProvider";
    private const string   EVENT_CLASS_NAME = "SongAdded";
    ...

    private void buttonAdd_Click(object sender, System.EventArgs e)
    {
        string  albumTitle;
        string  artistName;
        string  genre;
        int     albumId;
        SqlTransaction transaction;
        bool    submitEvents;
        EventCollector  eventCollector = null;

        // Extract values from the controls.
        albumTitle = this.textBoxAlbumTitle.Text;
        artistName = this.textBoxArtistName.Text;
        genre = (string) this.comboBoxGenre.SelectedItem;

        // Create a SQL transaction that will wrap adding the
        // album record and the individual song records.
        transaction = this.dbConnection.BeginTransaction();

        // Determine if we need to submit events based on the
        // state of the checkbox.
        submitEvents = this.checkBoxSubmitEvents.Checked;

        // If we're supposed to submit events for each song,
        // create an event collector.
        if (submitEvents)
        {
            eventCollector = new EventCollector(
                this.songAlertsApp,
                PROVIDER_NAME);
        }

        try
        {
            // Add the album record.
            albumId = AddAlbum(
            albumTitle,
            artistName,
            genre,
            transaction);

        // For each song specified, add a song record.
        foreach (string songTitle in Songs)
        {
            int songId = AddSong(
                songTitle,
                albumId,
                transaction);

            // If we're supposed to submit events, then create
            // an event record and submit it.
            if (submitEvents)
            {
                Event evt =
                    new Event(songAlertsApp, EVENT_CLASS_NAME);

                // Set the SongId event class field.
                evt["SongId"] = songId;

                // Write the event.
                eventCollector.Write(evt);
            }
        }

        // Commit the transaction.
        transaction.Commit();

        transaction = null;

        if (submitEvents)
        {
            // Commit the event batch.
            eventCollector.Commit();
            eventCollector = null;
        }

        // Clear the form now that the data has been added.
        ClearForm();
        }
        catch (Exception ex)
        {
            MessageBox.Show(
                ex.Message,
                "Unable to add songs",
                MessageBoxButtons.OK,
                MessageBoxIcon.Error);

            // Roll back the transaction.
            if (transaction != null)
            {
                try
                {
                    transaction.Rollback();
                }
                catch (Exception)
                {
                    // The rollback may throw an error if the
                    // transaction was already rolled back by
                    // a SQL error. In this case, just absorb
                    // the exception.
                }
            }

            if (submitEvents && eventCollector != null)
            {
                // Abort the event batch.
                eventCollector.Abort();
            }
        }
    }
    ...
}

This code reads the contents of the text boxes in the AddSongs user interface and adds the appropriate album and song records by calling the AddAlbum() and AddSong() helper methods. It checks the state of the check box and sets the value of a local Boolean variable, submitEvents, accordingly. If the submitEvents variable is true, the code constructs an EventCollector and uses the Event Object API to submit an event for each song added. The EventCollector is initialized with the provider name, SongAddedEmbeddedProvider (via the string constant, PROVIDER_NAME, defined near the top of the class). Events submitted by AddSongs will be associated with this provider name.

The key point here is that the same event submission API we used to built hosted event providers can be used in a standalone program as well. None of the SQL-NS event submission APIs are limited to the hosted provider infrastructure provided by the SQL-NS engine. Although this example showed the Event Object API, you can also use any of the event submission APIs in standalone programs.

To test the event submission capabilities of the AddSongs program, run it again and check the Submit Events for Songs Added box. Now add some song data and click the Add to Database button. This adds the song records and submits the corresponding events.

Verify that the events have been submitted by checking the events table in your SQL-NS application database. You will actually see the events submitted three times: AddSongs will submit them once, but because song records were actually added to the music store database, events will also be submitted by the SQLProvider and the scheduled custom hosted event provider. The events submitted by AddSongs should appear immediately, but the events from the other event providers may be delayed because of their scheduling intervals. As before, you can distinguish the events submitted by the various event providers by querying the NSEventBatchView, as described in the sidebar, "Querying the NSEventBatchView," (p. 280), earlier in this chapter.

Submitting Events with SQL Stored Procedures

All the event submission APIs used in the examples so far have submitted events from managed code. This section looks at the SQL-NS stored procedures that you can use to submit events from SQL code. As described in the "SQL Stored Procedures for Event Submission" section (p. 262), the SQL-NS compiler creates a dedicated set of stored procedures for each event class when it compiles the application. The code in this section uses the stored procedures created for the SongAdded event class in the music store application.

The stored procedures offer two modes of event submission. In the first mode, you start an event batch, add events to it one at a time, and then close the event batch. In the second mode, you invoke a stored procedure with an events query (much like the one used with the SQLProvider). The stored procedure runs the query and submits the results as a batch of events.

Listing 8.22 shows the use of the stored procedures that do one-at-a-time event submission.

Listing 8.22. Using SQL Stored Procedures to Submit Events One at a Time

USE MusicStore
GO

DECLARE @eventBatchId BIGINT
DECLARE @providerName NVARCHAR(255)

SELECT @providerName = N'SongAddedSPProvider'

EXEC [SongAlerts].[NSEventBeginBatchSongAdded] @providerName, @eventBatchId OUTPUT
EXEC [SongAlerts].[NSEventWriteSongAdded] @eventBatchId, 1
EXEC [SongAlerts].[NSEventWriteSongAdded] @eventBatchId, 2
EXEC [SongAlerts].[NSEventFlushBatchSongAdded] @eventBatchId, 2

The NSEventBeginBatchSongAdded stored procedure starts a batch of SongAdded events. It takes the provider name as input and returns the new event batch ID as output. Notice that the stored procedure names are qualified with the application's schema name, SongAlerts. Like all other application database objects, the SQL-NS compiler creates the event submission stored procedures in the application schema.

Each call to NSEventWriteSongAdded adds a new SongAdded event to the event batch. The first argument this stored procedure takes is the event batch ID. This code uses the value obtained as output from the NSEventBeginBatchSongAdded stored procedure call that created the batch. The remaining arguments to NSEventWriteSongAdded represent the fields of the event class. Because the SongAdded event class has only one field, SongId, this stored procedure takes only one additional argument beyond the event batch ID. The calls to NSEventWriteSongAdded in Listing 8.22 provide two hard-coded SongId values.

The call to NSEventFlushBatchSongAdded closes the event batch. It takes the event batch ID and a count of events as arguments.

You can find the code shown in Listing 8.22 in the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\StoredProcedureEventProviders\EventSubmission1.sql file. Open this file in Management Studio and execute it to test these stored procedures.

Listing 8.23 shows the stored procedure that executes an events query to submit a whole batch of events at once.

Listing 8.23. Using the SQL Stored Procedure to Submit a Batch of Events with an Events Query

USE MusicStore
GO

DECLARE @eventBatchId BIGINT
DECLARE @providerName NVARCHAR(255)

SELECT @providerName = N'SongAddedSPProvider'

EXEC [SongAlerts].[NSEventSubmitBatchSongAdded]
        @providerName,
        N'SELECT TOP 5 SongId FROM [Catalog].[SongDetails]',
        NULL

The NSEventSubmitBatchSongAdded takes an event provider name and two queries: an events query that returns event data and a post query that runs after the events query. In the particular example shown in Listing 8.23, there is no post query, so a NULL value is given for the third argument.

The events query shown is trivial: It just selects the top five songs in the SongDetails view. Of course, in a real application, the events query could be much more elaborate; I've used this simple example for clarity. Whatever the form of the events query, it must return a set of data with columns that match the schema of the event class. Because the SongAdded event class only has a SongId field, this query selects only the SongId column from the view.

The file C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\StoredProcedureEventProviders\EventSubmission2.sql contains the code shown in Listing 8.23. You can test this code by opening the file in Management Studio and executing it.

Tip

Although the examples here show the stored procedures invoked from simple scripts, they can also be used very effectively in triggers on database tables. By calling the event submission stored procedures in trigger code, you can easily attach event submission to database activities. For example, you could create an insert trigger on the Songs table in the music store catalog in which you call the event submission stored procedures to submit events for the inserted song records. This would cause an event to be submitted automatically for every song record, without an external event provider having to poll the table.


Cleanup: Preparing for the Next Chapter

In preparation for the work in the next chapter, you should clean up the SQL-NS instance and supporting files that we created in this chapter. Although we continue to develop the music store instance in future chapters, we will re-create the instance from a specific starting point rather than continue to work with it in its current form. Complete the following instructions to clean up the instance:

1.
From a Notification Services Command Prompt on your development machine, navigate to the music store sample's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run cleanup.cmd to remove the music store instance and all associated databases.


Summary

Event providers are components that connect SQL-NS applications to real-world event sources. SQL-NS provides three built-in event providers: the FileSystemWatcherProvider, the SQLProvider, and the AnalysisServicesProvider. SQL-NS also includes a set of APIs you can use to build your own custom event providers.

Event providers can run as components inside the SQL-NS engine or as standalone programs. Event providers running in the SQL-NS engine are called hosted event providers. Those that run as standalone programs are called standalone (or nonhosted) event providers.

To use an event provider in your application, you either choose one of the built-in event providers, or build a custom event provider. You then configure your application to use the event provider by declaring it in the <Providers> section of your ADF.

If you're building a custom hosted event provider, you can implement it either as a continuous event provider or as a scheduled event provider that the SQL-NS engine invokes periodically. Both types involve writing a class that implements a SQL-NS interface and then providing the configuration information required for the SQL-NS engine to load this class at runtime.

You can build a standalone event provider as a dedicated program, as part of a bigger program with another primary purpose, or as SQL code that runs in the database. The particular form you choose for your application depends largely on the event source.

In this chapter, we looked at several ways to build event providers for the music store application. Some of these made more sense than others for this particular application, but it was important to look at all of them. In future chapters, we will use only one event provider with the music store application.


Chapter 9. Content Formatters

IN THIS CHAPTER

  • Overview of Content Formatting

  • Preparing the SQL-NS Instance

  • Using the XsltFormatter

  • Building a Custom Content Formatter

When a SQL-NS application matches events and subscriptions, it produces rows of notification data. To turn these rows of data into actual notifications, the application's distributor passes them through a content formatter that transforms the raw data into a meaningful message for the target recipient.

This chapter is all about content formatters: what they do, how they work, and how to use them in your own applications. We'll take an in-depth look at the built-in content formatter that SQL-NS provides, the XsltFormatter. You've already seen this formatter used in the examples in previous chapters, but this chapter explains some of its more advanced features and how to customize it for your own applications. This chapter also shows you how to build a custom content formatter.


Overview of Content Formatting

The "Component Configuration and the Phases of Processing" section (p. 61) in Chapter 3, "The Simplest Notification Application: Stock Quotes," describes the stages in the SQL-NS processing pipeline. The final stage is distribution, which encompasses the formatting and delivery of notification messages. Within the distribution phase, content formatters do the work of formatting the raw notification data.

When defining a notification class, you specify which content formatter should be used to format notifications of that class. You can choose to use the XsltFormatter, a built-in SQL-NS component that you customize through configuration settings, or you can build your own custom content formatter. At runtime (during the distribution phase), the SQL-NS engine loads the content formatter you specify and passes it raw notification data. The formatter processes this data and returns formatted strings that then become the content of the final notification messages.

From the perspective of the SQL-NS engine, a content formatter is like a black box: Raw data goes in one end, and a formatted message comes out the other. What happens inside the box is not important to the engine. A content formatter can employ any formatting strategy it needs to get the job done.

Content Formatters and Distributors

As you may recall from Chapter 3, distribution is handled by the part of the SQL-NS engine called the distributor. The distributor constantly looks for new batches of notification data produced by the generator and processes them when they become available.

The distributor hosts content formatters and passes the raw notification data to them. The distributor takes the formatted messages output by the content formatters and invokes delivery protocol components to deliver them. This is shown in Figure 9.1. The distributor runs in the SQL-NS engine, and therefore all content formatters run in the engine, too.

Figure 9.1. The distributor and the components within it.

[View full size image]

Using Multiple Distributors

Even though an application can have more than one distributor, we still usually refer to "the distributor" in singular form. All distributors in an application are identical and perform the same functions. Think of each distributor as a clone of every other. Additional distributors serve as extra workers that share a common workload. The term "the distributor" refers to all the distributors in the application collectively.

You configure distributors in the <Distributors> element of your application's ADF. Having multiple distributors improves the performance of your application when each distributor runs on a separate machine. This gives each distributor dedicated hardware resources it can employ to format and deliver notifications. Because each distributor attempts to fully exploit the available resources of its host machine, it does not make sense to put multiple distributors on a single machine. The SQL-NS compiler prevents you from configuring more than one distributor on a single machine by raising a compile error if it finds two or more <Distributor> elements in the ADF with the same value for the <SystemName> subelement.

Only the Enterprise Edition of SQL-NS supports hosting components on multiple machines. If you are using Standard Edition, all components of the SQL-NS engine must run on a single machine, so you can configure only a single distributor in your ADF.

Content Formatter Input and Output

When you define a notification class, you specify a schema for the notification data. Using this schema, the SQL-NS compiler creates a notifications table and a notifications view in the application database. At runtime, your match rules generate notifications by inserting into the notifications view, providing values for the notification class fields. A trigger on the notifications view writes these values into the notifications table.

The data in the notifications table, along with some information about subscribers and target devices, form the input to the content formatter. When it processes a batch of notifications, the distributor reads each row from the notifications table and passes it to the content formatter.

You can augment the schema of your notification class by defining a set of computed fields in the ADF. Computed fields are similar to notification fields, but they are never materialized in the notifications table. Rather, computed field values are obtained (or "computed") by evaluating expressions when the distributor reads the notification data. The values of the computed fields are passed to the content formatter as input, along with the notification fields, subscriber data, and delivery device information. The examples you've seen in previous chapters have not used computed fields, but you will see them used in this chapter. The section "Adding Computed Fields" (p. 318) shows the declaration of computed fields.

The output from an invocation of a content formatter is a single string. This string is referred to as the notification body and will be passed to a delivery protocol for delivery to the subscriber. The notification body is the message that the subscriber will ultimately see. In forming the notification body, the content formatter can use all, or just portion of, the input data, as well as external data that it obtains on its own.

Formatting for Different Languages and Device Types

Different subscribers may want to receive their notifications in different languages. SQL-NS has support for content formatters that provide language-specific formatting. With each notification, SQL-NS tracks the target subscriber locale and passes this value to the content formatter when the notification needs to be formatted. The subscriber locale is a code that indicates a language and a specific country or region. For example, "en-US" is the subscriber locale code that refers to U.S. English, "en-ZA" refers to South African English, "fr-FR" refers to French as used in France, and "fr-CA" refers to French as used in Canada. The content formatter can use the subscriber locale code to choose an appropriate type of formatting for the given language and country or region.

Note

Refer to the "Subscriber Locales" topic in the SQL-NS Books Online for a complete list of all the supported subscriber locale codes. You can also obtain the list of supported locale codes programmatically using the SubscriberLocaleEnumeration class in the SQL-NS API, also documented in the SQL-NS Books Online.

In addition to the subscriber locale, it's also important to consider the target device type when formatting a notification. For example, a cell phone can receive notifications but is capable of displaying only a few lines of text. A desktop email client, on the other hand, can display longer notifications and render HTML. Because of the differences in display capabilities, it's a good idea to provide device-specific formatting in your content formatter. SQL-NS tracks the target device type for each notification and passes this to the content formatter in addition to the subscriber locale and the notification data.

Given the subscriber locale and the device type, the content formatter can make appropriate decisions about how to format the data. If you are building a custom content formatter, as described in the "Building a Custom Content Formatter" section (p. 334), you can use the subscriber locale and device type values to decide what formatting operations should be performed.

You can also do locale and device type-specific formatting if you're using the built-in XsltFormatter. You can write a different XSL transform for each language and device type your application supports, and the XsltFormatter will load the appropriate transform based on the locale and device type values it receives. More details on how to use the XsltFormatter this way are given in the "Using Locale and Device-Specific Transforms" section (p. 329).

For SQL-NS to pass the appropriate locale and device type information to the content formatter, you must ensure that your match logic provides the appropriate values for the DeviceName and SubscriberLocale columns when inserting into the notifications view. Recall that the match rule for the NewSongByArtist subscriptions in the music store application currently looks like this:

INSERT INTO [SongAlerts].[NewSong]
SELECT subscriptions.SubscriberId,
       N'DefaultDevice',
       N'en-US',
       songs.SongTitle,
       songs.ArtistName,
       songs.AlbumTitle,
       songs.GenreName
FROM   [SongAlerts].[SongAdded] events
JOIN   [Catalog].[SongDetails] songs
    ON events.SongId = songs.SongId
JOIN   [SongAlerts].[NewSongByArtist] subscriptions
    ON subscriptions.ArtistName = songs.ArtistName

The second and third fields in the select list specify the device name and subscriber locale, respectively. These values are inserted into the DeviceName and SubscriberLocale columns in the notifications view. The device name refers to a subscriber device record in the NSSubscriberDevices table, in the instance schema (the combination of the SubscriberId and DeviceName values effectively forms a foreign key into the NSSubscriberDevices table). The subscriber locale argument specifies a locale code.

When the distributor reads the notification data in preparation for formatting, it obtains the device type for each notification from the corresponding subscriber device record (identified by the subscriber ID and device name). The distributor passes the device type value, along with the locale (and the rest of the notification data), to the content formatter.

In the match rule we just examined, and all the other examples you've seen so far, the device name and the locale values have been hard-coded. This works if the application supports only one device and locale, but real applications usually need a more flexible strategy.

When designing an application, you must decide how the correct device and locale values will be obtained in the match rules. One common strategy is to allow subscribers to specify the desired device type and locale for their subscriptions. To implement this, you define fields in the subscription class for the device name and locale. When a subscriber creates a subscription, she can specify exactly which device she wants to use, and in what language she wants to receive the notification messages, by supplying values for these subscription fields. These are then stored in the subscriptions table, along with all the other subscription fields. The match rule can then obtain the stored device name and locale values from the subscriptions view (which is already part of the join). We use this strategy in the examples shown in this chapter.

Tip

The SQL-NS application model allows you to easily implement a more dynamic approach to selecting a subscriber device. Let's say that you want to support delivering notifications to different devices at different times of the day. For example, during work hours, the subscriber wants to receive notifications on her work email address, but after hours, she wants to receive them on her cell phone.

To implement this, you can store in a separate device profile table the various device names and the times of day at which they should be used. Instead of obtaining the device names from the subscriptions view, in your match rule, you join with the device profile table (using the current time of day to select the appropriate device row) and pass the device name to the notify function.

Formatting Digest and Multicast Notifications

As you may recall from Chapter 5, "Designing and Prototyping an Application," SQL-NS supports aggregation of notifications into digest and multicast groups (see "Aggregation Options: Digest and Multicast," p. 147 in Chapter 5). Digesting combines several notifications going to the same subscriber into a single message. Multicast detects when the same notification data is going to multiple subscribers, formats the data once, and then sends a copy of the formatted data to each subscriber. Multicast delivery enhances performance because it avoids unnecessarily formatting the same notification data over and over.

Note

Multicast delivery is available only in SQL-NS Enterprise Edition.

To format a digest message, SQL-NS passes several sets of notification data to the content formatter at once. The formatter must be able to handle more than one set of notification data at a time and aggregate it all into a single formatted message. This is shown in Figure 9.2. In this chapter, we'll look at how the built-in XsltFormatter and custom content formatters implement support for digesting.

Figure 9.2. Formatting digest notifications.

[View full size image]

When doing multicast delivery, the distributor passes a single row of notification data to the formatter and reuses the formatter's output to deliver additional notifications. This is illustrated in Figure 9.3. To the formatter, a call to format a multicast notification looks the same as a call to format a single notification. The only difference is in the distributor's behavior: It will not call the formatter again for additional notifications with the same data.

Figure 9.3. Formatting multicast notifications.

[View full size image]

Using a Content Formatter in a SQL-NS Application

In the <NotificationClass> element in the ADF, you declare the content formatter in the <ContentFormatter> subelement. In the examples in earlier chapters, you saw the <ContentFormatter> element used to declare the XsltFormatter. Each notification class can have only one content formatter, but different notification classes in the same application can use different content formatters.

When declaring a content formatter, you specify the content formatter class, assembly, and runtime arguments. The class can either be the XsltFormatter or the name of a class that implements a custom content formatter, as described in the "Building a Custom Content Formatter" section (p. 334). The assembly needs to be specified only if you're using a custom content formatter, as shown in the section "Declaring a Custom Content Formatter in the ADF" (p. 344). The runtime arguments specified in the declaration are passed to the content formatter when it starts. Each formatter defines its own set of arguments: the XsltFormatter has a set of arguments that it requires, and if you build a custom formatter, you determine the arguments that it needs.

Content Formatter Security

Because content formatters run within the SQL-NS engine, they run in the security context of the Windows account under which the engine runs. This is typically the SQL-NS service account (when the engine runs as a Windows service). As described in Chapter 2, "Getting Set Up," the engine should be configured to run under a low-privileged account, with only the minimum required database permissions.

To perform distributor operations in the database (including content formatting), the engine account needs to be a member of the NSDistributor database role. Alternatively, the engine account can be a member of the NSRunService role, which is a member of NSDistributor.


Preparing the SQL-NS Instance

As in previous chapters, we continue developing the music store application in this chapter. In preparation for the material in this chapter, we need to make some changes to the application and re-create the SQL-NS instance that hosts it. The changes we need to make include removing all the extraneous event providers added in Chapter 8, "Event Providers," and modifying the subscription classes and match rules to accommodate locale and device-specific formatting.

This section describes the specific changes that are needed and provides instructions on how to get the new instance set up. It also describes how we get events and subscriptions into the application for testing the code we add in this chapter.

Cleaning Up the Old Instance

Before you begin, make sure that you complete the cleanup steps from Chapter 8, described in the "Cleanup: Preparing for the Next Chapter" section (p. 304). This completely removes the instance created in Chapter 8 from your system and allows you to create a new instance to replace it.

Changes to the ADF

This section describes the changes to the ADF that need to be made before we start looking at content formatters. These changes enable the scenarios that highlight key features of the content formatting infrastructure in SQL-NS. Later in this chapter, there will be other changes to make.

Removal of Extra Event Providers

In Chapter 8, we added several event providers to the application to test different ways of submitting events, but most of these aren't needed. In fact, we really need only one event provider for the application to work. In this chapter, we remove all but the standalone event provider embedded in the AddSongs program. We use AddSongs to add song records and also to submit events.

To clean up the event providers, we remove the unnecessary <HostedProvider> and <NonHostedProvider> declarations in the ADF. Because we're using only the embedded event provider in AddSongs, only its <NonHostedEventProvider> declaration should remain in the <Providers> element of the ADF.

When we added the SQLProvider to the application in Chapter 8, we also added a chronicle that was used to store the last time the events query was run. Because this chronicle serves to support the SQLProvider, which we're not using anymore, we can remove this chronicle entirely in this chapter.

Storing Device and Locale Information in Subscriptions

The "Formatting for Different Languages and Device Types" section (p. 309) described how the locale and device information for a subscription can be stored in fields of the subscription class. This allows the subscriber to specify values for these fields, which the match rule can insert into the notifications view (instead of inserting hard-coded values as we have been doing). Implementing this in the music store application requires two modifications. First, we have to add locale and device name fields to the schemas of the subscription classes. Then we have to change the match rules to use the values of these fields from the subscription views.

Listing 9.1 shows the changes to the subscription class schemas.

Listing 9.1. Changes to the Subscription Class Schemas to Store Device and Locale Information

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
    <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
    <Schema>
      <Field>
        <FieldName>ArtistName</FieldName>
        <FieldType>NVARCHAR(255)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
      <Field>
        <FieldName>DeviceName</FieldName>
        <FieldType>NVARCHAR(255)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
      <Field>
        <FieldName>Locale</FieldName>
        <FieldType>NVARCHAR(10)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
    </Schema>
    ...
  </SubscriptionClass>
  <SubscriptionClass>
    <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
    <Schema>
      <Field>
        <FieldName>GenreName</FieldName>
        <FieldType>NVARCHAR(255)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
      <Field>
        <FieldName>DeviceName</FieldName>
        <FieldType>NVARCHAR(255)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
      <Field>
        <FieldName>Locale</FieldName>
        <FieldType>NVARCHAR(10)</FieldType>
        <FieldTypeMods>NOT NULL</FieldTypeMods>
      </Field>
    </Schema>
     ...
   </SubscriptionClass>
  </SubscriptionClasses>
  ...
</Application>

To each subscription class schema, two fields have been added: DeviceName and Locale. DeviceName refers to the name of a subscriber device, as stored in the subscriber devices table in the instance schema. The Locale field stores a subscriber locale code. The match rule inserts the values of these fields into the notifications view when it generates notifications. When the distributor picks up the notifications for formatting, it will obtain the device type from the subscriber device record, based on the value of the DeviceName field inserted into the notifications view, and pass this, along with the Locale value, to the content formatter.

Note

After making the changes shown in Listing 9.1, the subscription management interface (SMI) built in Chapter 7, "The SQL-NS Subscription Management API," will no longer work because it is not aware of the new subscription class fields. We will not maintain the SMI in this and the remaining chapters of this book because the SMI coding would distract from the new topics that these chapters cover. Instead, we'll use a T-SQL script to enter subscriptions for testing. However, it should be clear that none of the changes described here invalidate any of the general principles and techniques of SMI development described in Chapter 7. Given a good understanding of the material in Chapter 7, you should be equipped to make the required changes to the SMI should you choose to.

Listing 9.2 shows the corresponding modifications to the match rules, to accommodate the new fields added to the subscription classes.

Listing 9.2. Changes to the Match Rules to Pass the DeviceName and Locale Values from the Subscriptions View

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      <Schema>
        ...
      </Schema>
      <EventRules>
        <EventRule>
          <RuleName>MatchNewSongsByArtist</RuleName>
          <Action>
              INSERT INTO [SongAlerts].[NewSong]
              SELECT subscriptions.SubscriberId,
                     subscriptions.DeviceName,
                     subscriptions.Locale,
                     songs.SongTitle,
                     songs.ArtistName,
                     songs.AlbumTitle,
                     songs.GenreName
              FROM   [SongAlerts].[SongAdded] events
              JOIN   [Catalog].[SongDetails] songs
                  ON events.SongId = songs.SongId
              JOIN   [SongAlerts].[NewSongByArtist] subscriptions
                  ON subscriptions.ArtistName = songs.ArtistName
            </Action>
          <EventClassName>SongAdded</EventClassName>
        </EventRule>
      </EventRules>
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <Schema>
        ...
      </Schema>
      <ScheduledRules>
        <ScheduledRule>
          <RuleName>MatchNewSongsByGenre</RuleName>
          <Action>
            ...

            --Do the match.
            INSERT INTO [SongAlerts].[NewSong]
            SELECT subscriptions.SubscriberId,
                   subscriptions.DeviceName,
                   subscriptions.Locale,
                   songs.SongTitle,
                   songs.ArtistName,
                   songs.AlbumTitle,
                   songs.GenreName
            FROM   [SongAlerts].[SongAddedLog] eventlog
            JOIN   [Catalog].[SongDetails] songs
                ON eventlog.SongId = songs.SongId
            JOIN   [SongAlerts].[NewSongByGenre] subscriptions
                ON subscriptions.GenreName = songs.GenreName
            JOIN   [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes] chron
                ON subscriptions.SubscriptionId = chron.SubscriptionId
            WHERE   eventlog.TimeAdded &gt;= chron.LastProcessingTime
                AND eventlog.TimeAdded &lt; @CurrentProcessingTime

            ...
          </Action>
        </ScheduledRule>
      </ScheduledRules>
     ...
     </SubscriptionClass>
   </SubscriptionClasses>
  ...
</Application>

Because they are part of the subscription class schemas, DeviceName and Locale become columns in the subscription views that the match rules use. Instead of using hard-coded values for the DeviceName and SubscriberLocale columns in the notifications view, the match rules now use the values in the subscription views.

Adding Computed Fields

The "Content Formatter Input and Output" section (p. 309) describes computed fields that form part of the input to content formatters. Up to this point, none of the examples have used computed fields. To examine how computed fields work, we add one to the NewSong notification class. Listing 9.3 shows the code that defines a new computed field.

Listing 9.3. Declaration of a Computed Field in the NewSong Notification Class Declaration

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      <Schema>
        <Fields>
          ...
        </Fields>
        <ComputedFields>
          <ComputedField>
            <FieldName>NotificationDateTime</FieldName>
            <SqlExpression>
              CONVERT(NVARCHAR(255), GETDATE(), 100)
            </SqlExpression>
          </ComputedField>
        </ComputedFields>
      </Schema>
      ...
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

Computed fields are defined in the <ComputedFields> element, immediately following the <Fields> element in which the standard notification fields are declared. Each <ComputedField> declaration specifies a field name and a SQL expression that computes the value of the field.

To understand the kinds of expressions you can specify in the <SqlExpression> element, imagine a SELECT statement that selected all the fields defined in the notification class from the notifications table. A portion of this statement might look like

SELECT SongTitle, ArtistName, AlbumTitle, Genre
FROM NewSongNotifications...

Now imagine that in addition to the notification fields, you want the SELECT statement to return additional computed values in the resultset. For example, if you wanted to obtain an additional column that held the artist name in uppercase, you could modify the select statement as follows:

SELECT SongTitle, ArtistName, AlbumTitle, Genre, UPPER(ArtistName) AS UpperCaseArtistName
FROM NewSongNotifications...

The resultset from this statement would contain an additional column called UpperCaseArtistName that held the result of evaluating the SQL expression UPPER(ArtistName) for each row.

Note

UPPER() is a built-in SQL function that converts a string to uppercase.

This is how computed fields work. In the <SqlExpression> element of a computed field definition, you can specify any expression that can legally appear in the SELECT clause of a SQL query. The expression can involve constants, fields in the notification class, and calls to SQL functions. The distributor incorporates these expressions in the SELECT statement it uses to read rows from the notifications table, in the same way that the UPPER(ArtistName) expression was used in the previous SELECT statement example. In the computed field shown in Listing 9.3, the expression calls the built-in SQL CONVERT() and GETDATE() functions to get the current date and time in a readable format.

Computed fields provide a convenient way for you to use SQL functions to format some of the notification data, before the content formatter is even invoked. The distributor passes the values of the notification fields and the computed fields to the content formatter, so the results of the computed field expressions can be used to produce the final formatted notification.

Creating the New Instance

Supplementary files containing the code shown in this chapter are located in the C:\SQL-NS\Chapters\09\SupplementaryFiles directory:

  • ApplicationDefinition-12.xml contains the code changes described in the previous section and represents the starting point for the music store application in this chapter.

  • ApplicationDefinition-13.xml replaces the XsltFormatter with a custom content formatter that we build in the "Building a Custom Content Formatter" section (p. 334) later in this chapter.

  • NewSong-0.xslt contains the XSL transform used in examples in previous chapters to format the music store application's NewSong notifications with the XsltFormatter. This file is not used in this chapter; it is included only for reference.

  • NewSong-1.xslt contains a version of the previous XSL transform, augmented to use the computed field declared in Listing 9.3. In this chapter, we use this modified version of the XSL transform when formatting notifications with the XsltFormatter.

To work with the examples in this chapter, you'll need to re-create the music store instance on your system from scratch. Use the following instructions to prepare the source files and get the new instance running:

1.
If you have not already done so, clean up any old version of the music store instance left on your system, as described in the "Cleaning Up the Old Instance" section (p. 314).

2.
Make sure that the music store instance's ICF, C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml, contains the same code as the Chapter 8 supplementary file, C:\SQL-NS\Chapters\08\SupplementaryFiles\InstanceConfiguration-1.xml. If you worked through the examples in Chapter 8, your ICF should already contain the correct code. To be absolutely certain, you can copy over the supplementary file using the following command:

[View full width]

copy /y C:\SQL-NS\Chapters\08\SupplementaryFiles\InstanceConfiguration-1.xml C:\SQL-NS
  \Samples\MusicStore\InstanceConfiguration.xml

3.
Copy the supplementary file, C:\SQL-NS\Chapters\09\SupplementaryFiles\ApplicationDefinition-12.xml over the music store application's ADF, C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml, using the following command:

[View full width]

copy /y C:\SQL-NS\Chapters\09\SupplementaryFiles\ApplicationDefinition-12.xml C:\SQL-NS
  \Samples\MusicStore\SongAlerts\ApplicationDefinition.xml

4.
Copy the modified XSL transform in the supplementary file, C:\SQL-NS\Chapters\09\SupplementaryFiles\NewSong-1.xslt, over the following files in the application directory: C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\NewSong.xslt and C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\en-US\NewSong.xslt. Use the following commands:

[View full width]

copy /y C:\SQL-NS\Chapters\09\SupplementaryFiles\NewSong-1.xslt C:\SQL-NS\Samples
  \MusicStore\SongAlerts\XslTransforms\NewSong.xslt

[View full width]

copy /y C:\SQL-NS\Chapters\09\SupplementaryFiles\NewSong-1.xslt C:\SQL-NS\Samples
  \MusicStore\SongAlerts\XslTransforms\en-US\NewSong.xslt

Be sure to execute both copy commands so that the new XSL transform is copied to both locations. The purpose of the XSL transform in the second location is explained in the later section, "Directory Layout for XSL Transform Files" (p. 329).

5.
From a Notification Services Command Prompt on your development machine, navigate to the music store instance's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

6.
Run create_music_store_database.cmd. This rebuilds the music store database.

7.
Run create_with_argument_key.cmd to compile the instance and application from scratch.

8.
Run register_with_argument_key.cmd to register the instance.

9.
Run grant_permissions.cmd to assign the appropriate database permissions.

10.
Run enable.cmd to enable the instance.

11.
Start the instance's Windows service with the following command:

net start NS$MusicStore

Submitting Events

As described earlier, we will use the AddSongs program you saw in Chapter 8 to submit events. Refer back to the section "The AddSongs Program" (p. 243) in Chapter 8 for instructions on how to build and run the program. Because we want AddSongs to act as an event provider, check the Submit Events for Songs Added box whenever you use AddSongs in this chapter.

Adding Subscribers and Subscriptions

To test the code in this chapter, we need to add a set of subscribers and subscriptions that specify various locales and device types. Building a fully functional SMI to do this would be a distraction (most of the time would be spent building the appropriate user interface); instead, we'll use a simple T-SQL script to enter subscriber and subscription information.

Open the script, C:\SQL-NS\Chapters\09\Scripts\AddSubscribersAndSubscriptions.sql in Management Studio and examine the code in it. The script inserts subscriber, subscriber device, and subscription data into the views in the instance and application schemas. Run the script now to insert the data (you must have completed the instructions in the "Creating the New Instance" section [p. 319] before you do this).

Notice that the script creates one device called CellPhone and another called WorkEmail for each subscriber. The device names are tokens that can be displayed in an SMI to help users distinguish their devices. The device types for these devices are TextMessageDevice and Email, respectively.

Two NewSongByArtist subscriptions are created for each subscriber. Each subscription is associated with a different device: One specifies WorkEmail as the target device, and the other specifies CellPhone. Each subscription also specifies a locale for which the notification should be formatted. The variations in the locale and device data enable us to test the locale and device-specific formatting facilities of SQL-NS, later in this chapter.

Specifying Device Types

The locale codes come from a fixed set of allowed values. As mentioned earlier, the SQL-NS Books Online provides a complete list of the supported locale codes. The device type values, in contrast, do not come from a fixed list: You can choose any string to represent a device type. The only constraint is that your content formatter must understand the device type strings that you use. When building an application, you must decide on a set of device types you will support, and choose a string identifier for each. You must then use these strings consistently when setting up subscriber devices and implementing your content formatter. In this chapter's example, we've used the string TextMessageDevice to represent devices that can receive short text messages (such as cell phones or pagers) and Email for devices that can receive full email messages.

If you're using the XsltFormatter, you can configure it to choose an appropriate XSL transform based on your device type strings, as described in the section "Using Locale and Device-Specific Transforms" (p. 329). If you are writing a custom content formatter, you must write the code so that it properly interprets your device type strings. The "Building a Custom Content Formatter" section (p. 334) shows an example of this.

Later in this chapter, we will submit into the application events that match some of the subscriptions added in this section. We'll choose events that match subscriptions with a variety of locales and target device types to illustrate how the content formatter chooses the appropriate formatting.


Using the XsltFormatter

Although we used the XsltFormatter in previous chapters' examples, we haven't really looked at how it works in detail. You should already be familiar with its basic operation: The XsltFormatter converts notification data into XML and then applies an application-specific XSL transform to obtain a formatted notification message. This section examines the method that the XsltFormatter uses to form the XML representation of the notification data, the structure of the XSL transforms, and support for locale- and device type-specific formatting.

Note

Although we've used the XsltFormatter since the first prototype implementation, you should not think of it as just a prototype component. The XsltFormatter is a powerful tool used to do content formatting in many production notification applications.

Configuring the XsltFormatter in the ADF

When you declare the content formatter for a notification class, you can specify the formatter class name as XsltFormatter. This tells SQL-NS to use the built-in XsltFormatter to format notifications of that notification class. Listing 9.4 shows the <ContentFormatter> declaration in the NewSong notification class in the music store application's ADF.

Listing 9.4. The XsltFormatter Declaration in the NewSong Notification Class Declaration

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <ContentFormatter>
        <ClassName>XsltFormatter</ClassName>
        <Arguments>
          <Argument>
            <Name>XsltBaseDirectoryPath</Name>
            <Value>%_ApplicationBaseDirectoryPath_%\XslTransforms</Value>
          </Argument>
          <Argument>
            <Name>XsltFileName</Name>
            <Value>NewSong.xslt</Value>
          </Argument>
        </Arguments>
      </ContentFormatter>
      ...
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

The <ClassName> element specifies XsltFormatter, and the <Arguments> element supplies the two arguments required by the XsltFormatter:

  • XsltBaseDirectoryPath specifies the root directory from which the formatter should load XSL transforms.

  • XsltFileName specifies the name of the XSL transform file to load.

At runtime, the formatter combines the value of these two arguments to locate the XSL transform file.

The XsltFormatter also accepts a third, optional argument: DisableEscaping (not shown in Listing 9.4). Use of this argument is described in the following section.

Input to the XSL Transforms

For each notification that it formats, the XsltFormatter converts the notification data into an XML fragment. This XML fragment is the input to the XSL transform that the formatter applies.

To write XSL transforms for use with the XsltFormatter, you need to understand the format of the input XML. Unfortunately, no single schema describes the input XML. Rather, the XsltFormatter produces an XML structure that depends on the fields in the notification class. In general, the input XML is structured as follows:

<notifications xml:lang="... Subscriber Locale...">
    <notification>
        <NotificationField1Name>NotificationField1Value</NotificationField1Name>
        <NotificationField2Name>NotificationField2Value</NotificationField2Name>
        ...
        <NotificationFieldNName>NotificationFieldNValue</NotificationFieldNName>
        <ComputedField1Name>ComputedField1Value</ComputedField1Name>
        <ComputedField2Name>ComputedField2Value</ComputedField2Name>
        ...
        <ComputedFieldNName>ComputedFieldNValue</ComputedFieldNName>
    </notification>
    <notification>
        <NotificationField1Name>NotificationField1Value</NotificationField1Name>
        <NotificationField2Name>NotificationField2Value</NotificationField2Name>
        ...
        <NotificationFieldNName>NotificationFieldNValue</NotificationFieldNName>
        <ComputedField1Name>ComputedField1Value</ComputedField1Name>
        <ComputedField2Name>ComputedField2Value</ComputedField2Name>
        ...
        <ComputedFieldNName>ComputedFieldNValue</ComputedFieldNName>
    </notification>
    ...
</notifications>

The outermost element is <notifications>, which has an xml:lang attribute that specifies the locale for which the notification data should be formatted. The <notifications> element can contain one or more <notification> elements, each of which encapsulates the data from a single notification. This allows for digest notifications to be formatted together. When digesting is performed, the <notifications> element contains a <notification> element for each notification in the digest group. If digesting is turned off, the <notifications> element always contains a single <notification> element.

Note

Even with digesting turned on, the <notifications> element contains only a single <notification> element in the case of notifications that are not grouped with other notifications.

Within the <notification> element is a subelement for each field in the notification class schema (including the computed fields). These subelements have the same names as the notification fields, and their values are the notification field values. The following is an example of the XML generated for notifications of the music store application's NewSong notification class:

<notifications xml:lang="en-US">
    <notification>
        <ArtistName>Diana Krall</ArtistName>
        <Genre>Jazz</Genre>
        <SongTitle>I Love Being Here With You</SongTitle>
        <AlbumTitle>Live in Paris</AlbumTitle>
        <NotificationDateTime>May  7 2004  9:44PM</NotificationDateTime>
    </notification>
    <notification>
        <ArtistName>Diana Krall</ArtistName>
        <Genre>Jazz</Genre>
        <SongTitle>Let's Fall in Love</SongTitle>
        <AlbumTitle>Live in Paris</AlbumTitle>
        <NotificationDateTime>May  7 2004  9:44PM</NotificationDateTime>
    </notification>
</notifications>

This example shows two notifications digested together. In each <notification> element, there are subelements for the notification fields ArtistName, Genre, SongTitle, and AlbumTitle, and the single computed field, NotificationDateTime.

Note

It's important to pay attention to the casing of the element names in the XML (because XML element names are case sensitive). The <notifications> and <notification> element names are always given in lowercase. The names of the field elements within <notification> have the same casing as the field names defined in the notification class schema in the ADF.

When you understand the pattern that the XsltFormatter uses to construct its XML, you can easily work out what the XML will look like for any notification class. All you need to know is the list of fields in the notification class schema. If you plan to use the XsltFormatter in an application of your own, you will typically use this pattern to create the XML for some sample notifications by hand and use this to develop and test your XSL transforms. The section "Writing an XSL Transform for the XsltFormatter" (p. 326) provides more information on writing your own XSL transforms for use with the XsltFormatter.

Tip

An alternative exists to constructing a sample of the input XML by hand. You can configure the XsltFormatter to use a "pass-through" XSL transform that echoes the input XML without transforming it. Then feed some events and subscriptions into your application and observe the formatter output, as written to the output file by the File delivery protocol. This tells you exactly what XML the XsltFormatter is generating.

You can find the source code for a pass-through XSL transform in the file C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\PassThrough.xslt. You can specify this transform in the arguments to the XsltFormatter in any application.

If your notification data contains characters that have special meaning in XML, for example, > or <, the XsltFormatter replaces these characters with their equivalent escape sequences (also known as entity references) when forming the XML representation. For example, if the notification data contains the value Five>Four, the XsltFormatter turns this into Five&gt;Four. This ensures that the XML fragment is well formed.

You can use the optional DisableEscaping argument in the XsltFormatter declaration to control its escaping behavior. By default (if the DisableEscaping argument is not specified or set to false), the XsltFormatter replaces all XML reserved characters in the notification data with their equivalent escape sequences. With the DisableEscaping argument set to TRue, the XsltFormatter does not perform escaping, so the notification data passes through unaltered.

If your notification data contains valid XML fragments or escape sequences, set DisableEscaping to TRue so that the formatter places these directly into the XML it passes to the XSL transform. In all other cases, omit the DisableEscaping argument or set it to false.

Writing an XSL Transform for the XsltFormatter

This section looks at an example of an XSL transform that can be used to format notifications with the XsltFormatter. This transform takes the XML that the XsltFormatter generates from the notification data as input and produces a string that represents a formatted notification.

Note

XSL is a sophisticated language that cannot be covered fully in a single section of this book. The example shown here assumes some basic understanding of how XSL transforms work. If you are not familiar with XSL, many good reference books and online tutorials are available for you to consult. The complete XSL specification is available at http://www.w3.org/TR/xslt.

Listing 9.5 shows an XSL transform that produces a formatted notification from NewSong notification data.

Listing 9.5. An XSL Transform That Formats NewSong Notifications

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output method="text"/>

    <xsl:template match="notifications">
New songs are available for download from the music store!
        <xsl:apply-templates/>
Notification produced at: <xsl:value-of select="notification[1]/NotificationDateTime" />
        <xsl:text>
        </xsl:text>
    </xsl:template>

    <xsl:template match="notification">
        <xsl:text>
        </xsl:text>
         Song Title:  <xsl:value-of select="SongTitle" />
         Artist Name: <xsl:value-of select="ArtistName" />
         Album Title: <xsl:value-of select="AlbumTitle" />
         Genre:       <xsl:value-of select="Genre" />
    </xsl:template>
</xsl:stylesheet>

The transform is an XML document that conforms to the XSL schema. The root element is <xsl:stylesheet>, and within this element are subelements that describe how the input XML should be transformed.

The <xsl:output> element indicates that the transform outputs plain text. If you write a transform that outputs XML or HTML, you can specify the output type by changing the value of the method attribute on the <xsl:output> element to xml or html.

The <xsl:template> elements define templates that are matched against elements of the input XML. When the XSL processor encounters an element in the input XML, it looks for a matching template in the XSL transform and applies it. In the example shown in Listing 9.5, templates are defined for the <notifications> and <notification> elements of the input XML. Recall that <notifications> is the root element of the input XML. The template that matches <notifications> provides the overall structure for the final message. The template that matches <notification> defines the formatting for each individual notification within the final message.

The <notifications> template outputs a string that appears at the top of the final notification message. Think of this as a heading that appears before the text of the individual notifications. After displaying the heading message, the template specifies <xsl:apply-templates/> to instruct the XSL processor to match any subelements against the appropriate templates. In the final output, <xsl:apply-templates/> is replaced with the text that results from applying the rest of the templates in the XSL transform. After the <xsl:apply-templates/> instruction, the <notifications> template outputs a string that appears at the bottom of the notifications message. This string displays the time that the notification was produced, as given by the value of the NotificationDateTime computed field. Notice that this string selects the value of the NotificationDateTime field from the first <notification> child element.

The template that matches the <notification> element produces text for each individual notification. It displays labels and values for the fields in the notification, using the <xsl:value-of> element to select the values of the field subelements.

When applied to a notification, the transform shown in Listing 9.5 produces output that looks like this:

New songs are available for download from the music store!

        Song Title:  I Love Being Here With You
        Artist Name: Diana Krall
        Album Title: Live in Paris
        Genre:       Jazz

        Song Title:  Let's Fall in Love
        Artist Name: Diana Krall
        Album Title: Live in Paris
        Genre:       Jazz
Notification produced at: August  15 2005  5:47PM

Although the example shown here is simple, you can use the full power of XSL to write sophisticated transforms for your own applications. Many XML editors today provide rich support for designing XSL transforms and facilities for testing and debugging them with sample XML inputs. If you're developing a complex XSL transform, these tools can help you work out problems in isolation, before using the transform with SQL-NS. To test your XSL transform, you can either construct a sample XML input by hand using the pattern described in the section "Input to the XSL Transforms," p. 324, or obtain one using a pass-through XSL transform with the XsltFormatter, described in the tip near the end of that section.

Tip

The XsltFormatter caches XSL transforms in memory after reading them from disk the first time. Therefore, if you change the contents of the XSL transform file while the distributor is running, your change will not take effect. To force the XsltFormatter to reload the XSL transform from disk, stop and restart the SQL-NS Windows service.

Using Locale and Device-Specific Transforms

In the examples we've looked at so far, the XsltFormatter uses a single XSL transform to format all notifications. However, the XsltFormatter allows you to provide a different XSL transform for each locale and device type that your application supports. For example, you can write one transform for formatting notifications in U.S. English and another for formatting them in French. Each transform would output words in the right language and provide culture-specific formatting of dates, times, currencies, and other elements.

You can also provide separate transforms for different device types. You can have one transform that formats notifications for cell phones (returning a few lines of plain text as output) and another for email devices (returning HTML-formatted text). The XsltFormatter supports all combinations of locales and device types, so you could actually have separate XSL transforms for U.S. English cell phones, U.S. English email devices, French cell phones, and French email devices. In this section, we look at how to specify different transforms for different locales and device types and then test these transforms with the music store application.

Directory Layout for XSL Transform Files

As you've already seen, in the XsltFormatter declaration in the ADF, you provide an argument called XsltBaseDirectoryPath. This argument specifies the base directory in which the XSL transform files are located. To use specific transforms for different languages and locales, you need to create subdirectories beneath the base directory and place your transform files in them, according to a specific pattern. To understand the directory layout, you first need to understand the way the XsltFormatter locates XSL transforms at runtime.

For each notification, the distributor passes the XsltFormatter the values of the notification fields, as well as the locale and device type for which the notification should be formatted. The XsltFormatter uses the locale and device type values to form directory paths in which it looks for an XSL transform file. The XsltFormatter looks at the following sequence of directories:

  • <XsltBaseDirectoryPath>\<Locale>\<DeviceType>

  • <XsltBaseDirectoryPath>\<Locale>

  • <XsltBaseDirectoryPath>\<DeviceType>

  • <XsltBaseDirectoryPath>

In this list, <XsltBaseDirectoryPath> is the value of the XsltBaseDirectoryPath argument in the XsltFormatter declaration, <Locale> is the particular locale code in the notification data, and <DeviceType> is the device type string. For example, if the XsltBaseDirectoryPath is C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms, for a notification with a locale of en-US and a device type of Email, the XsltFormatter searches the following directories (in order) for the XSL transform file:

C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\en-US\Email
C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\en-US\
C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\Email
C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms\

In each of these directories, the XsltFormatter looks for an XSL transform file with the name specified in the XsltFileName argument in the content formatter declaration. It loads the transform from the first directory in which it finds it.

You can take advantage of this searching scheme by placing the XSL transforms for specific locales and device types in the appropriate directories. For example, if you wrote a transform to format notifications in French for the Email device type, you would create a directory below the XsltBaseDirectoryPath called fr-FR\Email and place the transform file in it. Similarly, if you wrote a transform for French notifications formatted for the TextMessageDevice device type, you would place it in a subdirectory called fr-FR\TextMessageDevice.

The nature of the searching algorithm used by the XsltFormatter allows you to provide default transforms for use when the formatter cannot find a specific transform for the particular locale and device type of a given notification. This is useful if you don't want to provide specific transforms for each and every locale and device type. For example, if you wanted to provide a single French transform for use regardless of device type, you could create an fr-FR directory without any device type subdirectories beneath it and place your transform there. The formatter's first attempt to load the transform will look for a devices-pecific subdirectory beneath fr-FR and fail to find it. It will then look in fr-FR itself and find the file there. Similarly, you can create a default transform for a particular device type that gets used when the formatter fails to find a locale-specific transform for that device type. By placing a transform in the root of the XsltBaseDirectoryPath, you can provide a default transform for when the formatter cannot locate either a locale- or device-specific transform.

Note

All the transform files in the various directories must have the same name. This name is the value you specify for the XsltFileName argument in the XsltFormatter declaration in the ADF.

Locale and Device-Specific Transforms for the Music Store Application

In this section, we look at several XSL transforms that format the music store's NewSong notifications for various locales and device types. With these transforms placed in files in the directory structure that the XsltFormatter requires, we test the application with some sample events and subscriptions that exercise locale and device-specific formatting.

Take a look at the C:\SQL-NS\Samples\MusicStore\SongAlerts\XslTransforms directory. This is the base directory for XSL transforms in the music store application. You will see a NewSong.xslt transform file in various subdirectories beneath the base directory:

  • en-US\Email\NewSong.xslt Formats notifications for U.S. English on Email devices

  • en-US\TextMessageDevice\NewSong.xslt Formats notifications for U.S. English on TextMessageDevice devices

  • en-US\NewSong.xslt Formats notifications for U.S. English, regardless of device type

  • fr-FR\Email\NewSong.xslt Formats notifications for French on Email devices

  • fr-FR\TextMessageDevice\NewSong.xslt Formats notifications for French on TextMessageDevice devices

  • fr-FR\NewSong.xslt Formats notifications for French, regardless of device type

There is also a NewSong.xslt in the root of the base directory, which will be used if no locale- or device-specific transform can be found.

Note

Earlier in this chapter, in step 4 of the instructions in the section, "Creating the New Instance," (p. 319) you copied the required XSL transform from a supplementary file into the XslTransforms base directory and its en-US sub-directory. This was necessary because previous chapters' examples relied on a different version of the XSL transform in those locations.

The other XSL transform files in the preceding list were already in place and did not require modification for this chapter (they were installed when you originally copied the book's source code in Chapter 2). These files were not used by the examples in previous chapters.

Let's examine the U.S. English transforms first. For email devices, we use the transform shown earlier, in Listing 9.5. This is also used as the default U.S. English transform and the default transform if no locale- or device-specific transform can be found. Thus, a copy of this transform exists in the en-US\Email, en-US, and base directories.

For TextMessageDevices, we need a transform that produces a smaller notification. Listing 9.6 shows this transform.

Listing 9.6. An XSL Transform That Produces Short NewSong Notifications

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output method="text"/>

    <xsl:template match="notifications">
New songs:
        Song Title:  <xsl:value-of select="notification[1]/SongTitle" />
        Artist Name: <xsl:value-of select="notification[1]/ArtistName" />
        Album Title: <xsl:value-of select="notification[1]/AlbumTitle" />
        Genre:       <xsl:value-of select="notification[1]/Genre" />
      <xsl:text>
      </xsl:text>
    </xsl:template>
</xsl:stylesheet>

Instead of displaying the data about every notification, this transform produces a message that contains just the data from the first notification. The heading message has also been shortened from that in Listing 9.5, and the message showing the notification time has been removed. This transform can be found in the en-US\TextMessageDevice directory.

For French notifications, we have a French equivalent of both the long U.S. English transform (shown in Listing 9.5) and the short one (shown in Listing 9.6). Listing 9.7 shows the long French transform, used for Email devices and as the default French transform.

Listing 9.7. An XSL Transform That Produces French NewSong Notifications

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output method="text"/>

    <xsl:template match="notifications">
Des nouvelles chansons sont disponibles pour téléchargement au magasin de musique!
        <xsl:apply-templates/>
Période d'avis: <xsl:value-of select="notification[1]/NotificationDateTime" />
        <xsl:text>
        </xsl:text>
     </xsl:template>

     <xsl:template match="notification">
        <xsl:text>
        </xsl:text>
        Titre de chanson: <xsl:value-of select="SongTitle" />
        Nom d'artiste:    <xsl:value-of select="ArtistName" />
        Album:            <xsl:value-of select="AlbumTitle"/>
        Genre:            <xsl:value-of select="Genre" />
    </xsl:template>
</xsl:stylesheet>

This transform can be found in the fr-FR\Email and fr-FR directories. It has the same basic structure as the transform in Listing 9.5, but the words have been translated into French.

Listing 9.8 shows the transform that produces shorter French notifications.

Listing 9.8. An XSL Transform That Produces Short, French NewSong Notifications

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output method="text"/>

    <xsl:template match="notifications">
Nouvelles chansons:
        Titre de chanson: <xsl:value-of select="notification[1]/SongTitle" />
        Nom d'artiste:    <xsl:value-of select="notification[1]/ArtistName" />
        Album:            <xsl:value-of select="notification[1]/AlbumTitle" />
        Genre:            <xsl:value-of select="notification[1]/Genre" />
       <xsl:text>
       </xsl:text>
    </xsl:template>
</xsl:stylesheet>

This has the same structure as the transform in Listing 9.6. You can find this transform in the fr-FR\TextMessageDevice directory.

Testing the Music Store Application with Locale- and Device-Specific Transforms

The code in the initial ADF you used to create the music store application (earlier in this chapter) configured the XsltFormatter with the appropriate XsltBaseDirectoryPath and XsltFileName arguments for it to work with the transforms described in the previous section. Therefore, no code changes or updates are needed before testing.

To see the locale and device-specific transforms in use, perform the following instructions:

1. | Make sure that you've created the instance as described in the "Creating the New Instance" section (p. 319) earlier in this chapter.
2. | If you haven't already done so, add the subscriber and subscription data as instructed in the "Adding Subscribers and Subscriptions" section (p. 321).
3. | Start the AddSongs program (located in the C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs\bin\debug directory). If you had the program open from the previous chapter, before you deleted and re-created the music store database, close and reopen it now. If you have not previously built the AddSongs program, refer to the section "The AddSongs Program" (p. 243) in Chapter 8 for instructions on how to do this.
4. | When AddSongs starts, connect to your SQL Server. If you select SQL Server Authentication, enter the SQL username and password of your development account.
5. | In the main AddSongs dialog box, check the Submit Events for Songs Added box.
6. | Enter the following song data (make sure that you specify the artist name exactly as it appears here; otherwise, the events won't match the subscriptions): Album Title: Live in Paris Artist Name: Diana Krall Genre: Jazz Song 1: I Love Being Here With You Song 2: Let's Fall in Love
7. | Click the Add to Database button.
8. | When the form clears, enter the following song data (make sure that you specify the artist name exactly as it appears here; otherwise, the events won't match the subscriptions): Album Title: Round About Midnight Artist Name: Miles Davis Genre: Jazz Song 1: 'Round Midnight Song 2: All of You
9. | Wait about 30 seconds and then check the notifications output file, C:\SQL-NS\Samples\MusicStore\FileNotifications.txt.

The notifications output file will contain various notifications formatted for the locales and device types specified in the subscriptions. You can refer back to the subscription data to verify that these appear correctly. Also, notice that the value of the computed field we defined in Listing 9.3, NotificationDateTime, has been included in the notifications formatted by the XSL transforms that reference it.

Note

The text produced by the XSL transforms is in UTF-8 encoding. If you open the notifications output file in a text editor configured to use another encoding, some of the characters in the French notifications may not appear correctly.


Building a Custom Content Formatter

As it does for event providers, SQL-NS offers a plug-in model for content formatters. If the built-in XsltFormatter doesn't meet your needs, you can easily build a custom content formatter to replace it.

This section looks at some of the reasons for building a custom content formatter and then describes the technical details of content formatter development. To illustrate the concepts with a practical example, we'll build a custom content formatter for the music store application.

Why Build a Custom Content Formatter?

Although the XsltFormatter does present a simple, convenient way to format notifications without writing custom code, the use of XSL transforms has some inherent limitations:

  • Beyond a basic level of complexity, XSL transforms become difficult to read, maintain, and debug.

  • XSL transforms do not provide an elegant way to do computations or iterative processing on the input data.

  • Accessing external data from an XSL transform is not straightforward.

  • It can be difficult to manage and maintain all the separate transform files required to do locale- and device-specific formatting.

If you build a custom content formatter, these limitations fall away.

Building a custom content formatter involves writing a class in managed code. With the expressiveness of the .NET languages and the full power of the .NET Framework at your disposal, implementing complex formatting in custom content formatter code tends to be easier than writing the equivalent XSL transforms.

In custom content formatter code, you have a great deal of flexibility in the way you can work with the input data. Using familiar programming constructs, you can perform computations such as calculating averages or totals, conditional processing, and iterative processing. Although it is possible to do all these things in XSL transforms, doing them in a procedural programming language is usually more intuitive for most developers.

One thing that almost always necessitates building a custom content formatter is the need for access to external data in the formatting process. In many SQL-NS applications, the notification data contains references to other data, stored externally, that the formatter needs to access to produce the final formatted notifications. For example, the notification data might contain a customer ID that refers to a customer record in an external database. When formatting such a notification, the formatter may need to access the customer record to obtain the customer's name. It's difficult to access external data in an XSL transform, but if you build a custom content formatter, you can use any of the data access APIs in the .NET Framework to do this.

In a custom content formatter, you are free to implement locale- and device-specific formatting in any way you choose. When the distributor invokes your formatter code, it passes you the locale and device type for which the notification should be formatted. You can use this information to make decisions about the final format of the string that your formatter returns. In a custom content formatter, the formatting code for each locale and device type is typically defined in the same source file. This makes it much easier to maintain than the separate transforms used by the XsltFormatter.

The Content Formatter Interface

To create a custom content formatter, you write a class that implements the IContentFormatter interface. This interface defines methods that the distributor uses to interact with content formatters. Listing 9.9 shows the definition of IContentFormatter.

Listing 9.9. The IContentFormatter Interface

public interface IContentFormatter
{
    void Initialize(StringDictionary arguments, bool digest);

    string FormatContent(
        string            subscriberLocale,
        string            deviceTypeName,
        RecipientInfo     recipientInfo,
        Hashtable[]       rawContent);

    void Close();

}

Note

IContentFormatter is in the SQL-NS namespace (Microsoft.SqlServer.NotificationServices) and is defined in the SQL-NS assembly. Any project that implements IContentFormatter must reference the SQL-NS assembly.

In your formatter class, you implement each of the IContentFormatter methods for your chosen formatting scheme. You then compile the formatter class into an assembly. To use the custom content formatter in a SQL-NS application, you specify the name of the formatter class and its containing assembly in a <ContentFormatter> element in the ADF. On startup, the distributor loads the assembly and creates an instance of the class.

At the start of processing, the distributor calls the content formatter's Initialize() method. The distributor passes Initialize() a dictionary of content formatter arguments, as specified in the content formatter declaration in the ADF. It also passes a Boolean flag indicating whether digesting is turned on.

For each notification (or set of notifications, in the case of digesting) that the distributor needs to format, it calls the FormatContent() method on the custom content formatter class. In the call to FormatContent(), the distributor passes the subscriber locale, device type, information about the recipient, and the raw notification data. The recipient information is an object of the RecipientInfo class (defined in the SQL-NS API) that has properties that return the subscriber ID and device address. The raw notification data is passed as an array of hash tables. There is one hash table in the array for each row of notification data that the formatter needs to process. Each hash table contains the values of the notification and computed fields in one row of notification data. The FormatContent() method returns a string that represents the formatted notification body. The distributor passes the string returned from FormatContent() to the delivery protocol for delivery.

Tip

The Hashtable class, used in the FormatContent() method signature, is defined in the System.Collections namespace. If you are writing your own class that implements IContentFormatter, you must reference this namespace in a using declaration.

The distributor calls the Close() method at the end of processing. In the Close() method, the content formatter should release any resources it is holding and prepare to shut down.

The distributor's contract with the content formatter guarantees that Initialize() will be called first, followed by a series of calls to FormatContent() (one for each notification or group of notifications that need to be formatted), followed by a call to Close(). FormatContent() will not be called after Close() unless Initialize() is first called again.

Implementing the Content Formatter Interface

To implement IContentFormatter, you write a class that provides implementations for the Initialize(), FormatContent(), and Close() methods. In this section, we implement a custom content formatter for the music store application's NewSong notifications. This formatter produces messages similar to the ones produced with XSL transforms in the previous section, but performs some additional processing to streamline the notification content.

Use the following instructions to open the Visual Studio solution containing the content formatter implementation:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\NewSongNotificationFormatter directory.
2. | Open the solution file NewSongNotificationFormatter.sln in Visual Studio.

The code for the content formatter class is in the NewSongNotificationFormatter.cs file. Listing 9.10 shows the implementation of the IContentFormatter methods in the content formatter class.

Listing 9.10. The NewSongNotificationFormatter Class

namespace SongAlertsCustomComponents
{
    class NewSongNotificationFormatter : IContentFormatter
    {
        private bool digest;

        public void Initialize(StringDictionary arguments, bool digest)
        {
            this.digest = digest;
        }

        public string FormatContent(
            string          subscriberLocale,
            string          deviceTypeName,
            RecipientInfo   recipientInfo,
            Hashtable[]     rawContent)
        {
            string body;

            switch(subscriberLocale)
            {
                case "en-US":
                    body = FormatUSEnglish(deviceTypeName, rawContent);
                    break;
                case "fr-FR":
                    body = FormatFrench(deviceTypeName, rawContent);
                    break;
                default:
                    body = FormatUSEnglish(deviceTypeName, rawContent);
                    break;
            }

            return body;
        }

        public void Close()
        {
            // Do nothing.
        }

        ... helper methods...
      }

}

The purpose of the Initialize() method is to set up the content formatter for the processing it will need to do later. In the Initialize() method, the formatter typically processes the arguments it is given (the argument names and values are taken from the content formatter declaration in the ADF) and stores their values in private member variables. Our content formatter does not take any arguments, so the code in Listing 9.10 ignores the arguments parameter.

The digest parameter to Initialize() indicates whether digesting is turned on. At first, the need for this parameter may not be apparent. After all, the formatter can always tell whether to perform digesting by the number of notifications that it gets passed in each call to the FormatContent() method. To understand the need for the digest parameter, imagine a case in which FormatContent() is called with just one notification. This could happen for one of two reasons: Either digesting is turned off, or, even though digesting is turned on, the particular notification passed to FormatContent() is a singleton (there are no other notifications in the batch that satisfy the digesting criteria to be grouped with this notification). The digest parameter allows the content formatter to distinguish between these two situations. The distinction is important for formatters that always apply special formatting when digesting is turned on, even if the digest group contains just a single notification.

Note

Our formatter saves the value of the digest parameter in a private member variable. As we look at the formatting code later, you'll notice that this member variable isn't actually used. I've included the code that initializes its value in Initialize() simply to illustrate a pattern that some formatters follow.

Whatever initialization is done in the Initialize() method, the Close() method should do the corresponding shutdown operations. For example, if Initialize() opens a database connection, Close() should close that connection. Because we have not created any resources in Initialize() that require explicit cleanup, and because our FormatContent() implementation is stateless, the Close() method does nothing.

The FormatContent() method implements the actual formatting operations. The code in Listing 9.10 uses the value of the subscriber locale to select a helper method to which it delegates the formatting task. Our formatter explicitly supports U.S. English and French formatting, so it has helper methods for these locales. The U.S. English helper method is used as a default in the case that the subscriber locale is not either en-US or fr-FR.

Listing 9.11 shows some of the helper methods in the content formatter class that are used to format U.S. English notifications. The source file also contains the equivalent helper methods for French notifications, although I have not shown them here. The French methods have the same structure as the U.S. English ones.

Listing 9.11. Helper Methods in the Content Formatter Class

[View full width]
    class NewSongNotificationFormatter : IContentFormatter
    {
        ... IContentFormatter methods ...

        private string FormatUSEnglish(
            string      deviceTypeName,
            Hashtable[] rawContent)
    {
            string body;

            switch (deviceTypeName)
            {
                case "Email":
                    body = FormatUSEnglishEmail(
                        deviceTypeName,
                        rawContent);
                    break;
                case "TextMessageDevice":
                    body = FormatUSEnglishTextMessageDevice(
                        deviceTypeName,
                        rawContent);
                    break;
                default:
                    body = FormatUSEnglishEmail(
                        deviceTypeName,
                        rawContent);
                    break;
            }

            return body;

          }

          private string FormatUSEnglishEmail(
              string      deviceTypeName,
              Hashtable[] rawContent)
          {
              StringBuilder body = new StringBuilder();
              int notificationCount = rawContent.Length;

              // Display the appropriate heading message,
              // depending on the number of notifications.
              if (notificationCount == 1)
              {
                  body.Append("There is a new song available for download from the music
   store!\n");
         }
         else
         {
            body.AppendFormat(
                "There are {0} new songs available for download from the music store!\n",
                 notificationCount);
         }

         // Extract the albums, artist names, and genres
         // from the notification data into parallel array lists.

         ArrayList albums = new ArrayList();
         ArrayList artists = new ArrayList();
         ArrayList genres = new ArrayList();

         foreach (Hashtable content in rawContent)
         {
             string album = (string) content["AlbumTitle"];
             string artist = (string) content["ArtistName"];
             string genre = (string) content["Genre"];

             if (!albums.Contains(album))
             {
                 albums.Add(album);
                 artists.Add(artist);
                 genres.Add(genre);
             }
         }

         // For each album, display the album name and artist name,
         // then show the list of songs.
         for (int i = 0; i < albums.Count; i++)
         {
              string album = (string) albums[i];
              string artist = (string) artists[i];
              string genre = (string) genres[i];

              body.Append("\n");
              body.AppendFormat("Album Title: {0}\n", album);
              body.AppendFormat("Artist Name: {0}\n", artist);
              body.AppendFormat("Genre: {0}\n", genre);
              body.Append("Songs:\n");

              foreach (Hashtable content in rawContent)
              {
                  if ((string) content["AlbumTitle"] == album)
                  {
                      body.AppendFormat(
                          "\tSong Title: {0}\n",
                         content["SongTitle"]);
                  }
              }
         }

         return body.ToString();
      }

      private string FormatUSEnglishTextMessageDevice(
          string      deviceTypeName,
          Hashtable[] rawContent)
      {
          StringBuilder body = new StringBuilder();
          int notificationCount = rawContent.Length;

          // Display the appropriate heading message,
          // depending on the number of notifications.
          if (notificationCount == 1)
          {
              body.Append("New song available:\n");
          }
          else
          {
              body.AppendFormat(
                  "{0} new songs available, including:\n",
                  notificationCount);
          }

          // Display the data for one song.
          body.Append("New songs:\n");
          body.AppendFormat("Song Title: {0}\n",
              rawContent[0]["SongTitle"]);
          body.AppendFormat("Artist Name: {0}\n",
              rawContent[0]["ArtistName"]);
          body.AppendFormat({"Album Title: {0}\n",
              rawContent[0]["AlbumTitle"]);
          body.AppendFormat("Genre: {0}\n",
              rawContent[0]["Genre"]);

          return body.ToString();
      }

      ... more helper methods ...
   }
}

The first helper method, FormatUSEnglish(), is the one called from FormatContent() when the subscriber locale is en-US, or anything other than fr-FR. This method looks at the device type (passed down from FormatContent()) and calls either FormatUSEnglishEmail() (to format a U.S. English notification for an Email device) or FormatUSEnglishTextMessageDevice() (to format a U.S. English notification for a TextMessageDevice device). Recall that device type strings can be any value you choose, as long as the content formatter understands them. Here the content formatter code is written to expect the strings we used when setting up subscriber devices.

Let's examine the FormatUSEnglishEmail() method (code shown in Listing 9.11). This method produces a formatted notification suitable for delivery to Email devices. The code begins by creating an object of the StringBuilder class (a .NET Framework utility class) that it uses to construct the formatted notification string incrementally.

By looking at the number of elements in the rawContent array of hash tables (each hash table contains the data from a single notification row), FormatUSEnglishEmail() determines the number of notifications it will need to format. Based on the number of notifications, it appends an appropriate heading message to the string builder. This results in a heading message that indicates the number of songs available for download. This is something that would have been more cumbersome to implement in an XSL transform.

After appending the heading message, the code performs some analysis on the input notification data to look for songs that have common album titles, artist names, and genres. In many cases, the notifications in a digest group represent different songs in a single album, by a single artist. All these songs would have the same genre. In the messages produced by the XSL transforms we used with the XsltFormatter, the album title, artist name, and genre were repeated for each song. Our custom content formatter attempts to streamline the notification messages by writing the album title, artist name, and genre once and then listing the song titles to which they apply beneath them.

This is achieved by constructing three parallel arrays from the notification data. The first array contains all the unique album titles in the set of notification data, and the second and third arrays contain the corresponding artist names and genres for those albums. Notice how the code refers to the notification data in the hash tables by using the index operator ([]) with the notification field names. After it has constructed the arrays, the code loops through the album's array and appends the information for each album to the string builder. It writes the album title, artist name, and genre, and then lists the songs individually. The song list is obtained by looping through the notification hash tables and appending the data from those in which the album title matches the current album title. In this way, all the essential notification data is eventually displayed without the unnecessary repetition. Again, the kind of analysis required to achieve this would have been difficult to do in an XSL transform but is fairly straightforward when writing C# code.

The FormatUSEnglishTextMessageDevice() method produces a smaller notification message, suitable for display on devices such as cell phones or pagers. Like the previous method, it uses a string builder to construct the formatted notification. This method produces a heading that indicates the number of notifications available and then displays the content of the first notification only.

After taking a look at the code, build the content formatter project in Visual Studio. This produces the content formatter assembly in the bin\Debug subdirectory of the NewSongNotificationFormatter project directory.

Declaring a Custom Content Formatter in the ADF

To use the custom content formatter in the music store application, we have to declare it in the ADF. Because each notification class can declare only one content formatter, the declaration of the custom content formatter replaces the XsltFormatter declaration in the NewSong notification class. Listing 9.12 shows the custom content formatter declaration.

Listing 9.12. Declaration of the Custom Content Formatter in the ADF

[View full width]
<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <ContentFormatter>
        <ClassName>SongAlertsCustomComponents.NewSongNotificationFormatter</ClassName>
        <AssemblyName>%_ApplicationBaseDirectoryPath_%\CustomComponents
  \NewSongNotificationFormatter\bin\Debug\NewSongNotificationFormatter.dll</AssemblyName>
      </ContentFormatter>
      ...
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

Like the previous declaration of the XsltFormatter, the custom content formatter is declared in a <ContentFormatter> element. The class name specifies the name of the content formatter class that we just implemented. (Note that the class name is always given with the namespace prefix.) The <AssemblyName> element specifies the assembly in which the content formatter class is implemented. The <AssemblyName> element was not required for the built-in XsltFormatter, but it is required for the custom content formatter. The value specified points to the assembly produced by building the content formatter project in Visual Studio. Because our content formatter does not take any arguments, the <Arguments> element is omitted. If you build a custom content formatter that does require arguments, you can specify them in your content formatter declaration, using the same syntax used for the XsltFormatter arguments.

Testing the Custom Content Formatter

To test the custom content formatter, we'll feed some events into the application designed to match the subscriptions we added earlier. As in the previous test we did, these events will produce notifications for both supported languages and device types.

To see the custom content formatter in use, perform the following instructions:

1. | Make sure that you've built the custom content formatter project in Visual Studio.
2. | Add the custom content formatter declaration shown in Listing 9.12 to your ADF. You can find this code in the supplemental ADF, ApplicationDefinition-13.xml, in C:\SQL-NS\Chapters\09\SupplementaryFiles.
3. | Update your instance using the same steps you used in the previous chapter. (For reference, see steps 27 in the "Testing the FileSystemWatcherProvider in the Music Store Application" section, p. 251, in Chapter 8.) Because the instance uses argument encryption, you must use update_with_argument_key.cmd, not update.cmd, to invoke nscontrol update.
4. | Start the AddSongs program.
5. | Check the Submit Events for Songs Added box.
6. | Enter the following song data (make sure that you specify the artist name exactly as it appears here; otherwise, the events won't match the subscriptions): Album Title: All for You Artist Name: Diana Krall Genre: Jazz Song 1: I'm an Errand Girl for Rhythm Song 2: Gee Baby, Ain't I Good to You
7. | Click the Add to Database button.
8. | When the form clears, enter the following song data (make sure that you specify the artist name exactly as it appears here; otherwise, the events won't match the subscriptions): Album Title: The Essential Miles Davis Artist Name: Miles Davis Genre: Jazz Song 1: Summertime Song 2: Time After Time
9. | Wait about 30 seconds and then check the notifications output file, C:\SQL-NS\Samples\MusicStore\FileNotifications.txt.

Unless you deleted the file after the previous test run, the output file will still contain the notifications produced by the XsltFormatter. After these notifications, you should see the new notifications produced by the custom content formatter. Notice how much cleaner the notifications look as a result of the streamlining that the custom content formatter performed.

Cleanup: Preparing for The Next Chapter

In preparation for the work in the next chapter, you should clean up the SQL-NS instance and supporting files created in this chapter. In the next chapter, we will re-create the instance before continuing to develop it. Complete the following instructions to clean up the instance:

1.
From a Notification Services Command Prompt on your development machine, navigate to the music store sample's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run cleanup.cmd to remove the music store instance and all associated databases.


Summary

In SQL-NS applications, the distributor handles the formatting and delivery of notifications generated by the matching logic. The distributor employs components called content formatters to do the actual formatting work.

Content formatters receive the raw notification data as input. This data consists of the notification fields and computed fields defined in the notification class. Along with the notification data, the distributor also passes the content formatter information about the locale and device type for which the notification should be formatted. The content formatter can use this information to provide locale- and device-specific formatting. The output of the content formatter is a single string that represents the notification body. The distributor passes this string to the delivery protocol for delivery to the subscriber.

In your applications, you can choose to use the built-in XsltFormatter, or you can build a custom content formatter. If you use the XsltFormatter, you specify application-specific formatting instructions through XSL transforms. You can specify a different transform for each locale and device type that your application supports. If you build a custom content formatter, you can implement the formatting operations in procedural code. It is often easier to implement complex formatting operations by writing procedural code than by writing XSL transforms. In a custom content formatter, you can also access external datasomething that is difficult to do from an XSL transform.

To build a custom content formatter, you develop a class that implements the IContentFormatter interface. You compile this class into an assembly and declare it in your ADF. At runtime, the distributor loads this class and passes it the notification data to format. In this chapter we built a custom content formatter for the music store application's notification class. By doing this, we were able to produce more meaningful, streamlined notifications than we could with the built-in XsltFormatter.


Chapter 10. Delivery Protocols

IN THIS CHAPTER

  • Overview of Notification Delivery in SQL-NS

  • Preparing to Work with Delivery Protocols

  • Using the Built-in SMTP Delivery Protocol

  • Building a Custom Delivery Protocol

  • Restoring Your System

Inside the distributor, notifications are formatted and then delivered. Formatting is handled by content formatters, as described in Chapter 9, "Content Formatters." This chapter looks at delivery protocols: the components that send formatted notification messages to external delivery systems for routing to subscribers.

This chapter explains how delivery protocols fit into the SQL-NS application model and how to use them in your own applications. It covers the key concepts behind delivery protocol operation, shows how to leverage the built-in SQL-NS delivery protocols, and explains how to build custom delivery protocols for your applications.


Overview of Notification Delivery in SQL-NS

An important part of designing a SQL-NS application is deciding how notifications will be sent to subscribers. You may choose to send notifications via email, instant messaging, or Short Message Service (SMS), for example. These are just a few possibilities; the SQL-NS framework enables you to support almost any delivery method you choose.

A single SQL-NS application may use multiple notification delivery methods. When building an application, you must decide which delivery methods make sense for the subscribers you're targeting.

After you decide on a set of delivery methods you want to support, you need to obtain delivery protocol components that implement those delivery methods. SQL-NS provides two built-in delivery protocols: the File protocol that we've been using in the prototype until now, and the SMTP protocol that we look at in this chapter. The File protocol is unlikely to be useful in a production application, although, as you've seen, it is useful in the prototype stage. The SMTP protocol, however, is commonly used in production applications. It implements the Simple Mail Transfer Protocol (SMTP) to send notifications via email. If email is one of the delivery methods you want to enable, you can use the built-in SMTP protocol.

If your application needs to support another delivery method, you have two choices: You can either implement a custom delivery protocol yourself (as explained later in this chapter), or you can obtain a custom delivery protocol from a vendor. Implementing a custom delivery protocol yourself is easy. As with other SQL-NS custom components, you develop a class that implements a SQL-NS interface. Because custom delivery protocols developed this way are reusable components, there is a growing community of independent software vendors building and licensing them for general use in SQL-NS applications. Today, various vendors have already made available delivery protocols that support delivery via SMS, fax, and voicemail. If they fit your needs, you can leverage these prebuilt custom delivery protocols instead of writing your own. As the SQL-NS development community grows, additional delivery protocols will certainly be implemented. You can find a list of Microsoft's partners that provide delivery protocol implementations and other delivery services at http://www.microsoft.com/sql/ns/partners/delivery.asp.

This section describes the basic concepts behind the delivery protocol infrastructure in SQL-NS. As a SQL-NS application developer, it's important to understand these concepts, whether or not you intend to build a custom delivery protocol.

Delivery Protocols in the SQL-NS Engine

In the SQL-NS engine, the distributor handles the formatting and delivery of notifications. The distributor hosts both content formatters and delivery protocols and forms a distribution pipeline for the notification data, as shown in Figure 10.1.

Figure 10.1. The notification distribution pipeline.

[View full size image]

In this pipeline, the content formatter turns the raw notification data into a readable message, and the delivery protocol sends this message out to an external delivery system.

To send the message, the delivery protocol may need to wrap it with additional information required by the delivery system. For example, if the delivery protocol sends email, it adds a number of headers, specifying the recipient's email address, the subject line, the content type (text or HTML), and other information, before sending the message to a mail server. To draw a simple analogy, you can think of the content formatter output as a letter; the delivery protocol places this letter in an envelope, writes the destination address on it, and then sends it out. The nature of the envelope and the form of the address information vary per delivery protocol.

As described in Chapter 9, an application can have multiple distributors, but each one is an identical copy of the others. Each distributor hosts its own copy of the delivery protocol components, which it uses to deliver the notifications it handles. When we use the term "the distributor," we are referring to the set of distributors in an application collectively.

Configuring Delivery Endpoints: Delivery Channels

The most often confused concept in SQL-NS is the distinction between delivery protocols and delivery channels. This distinction was called out in the section, "Delivery Protocols and Delivery Channels," (p. 99) in Chapter 4, "Working with SQL-NS Instances," but it's worth revisiting here.

A delivery protocol is a component that is hosted by the SQL-NS engine and implements the communication scheme required to send notifications to a particular type of delivery system. This communication scheme usually consists of a network protocol that specifies a message format and sequence of exchanges. For example, the built-in SMTP delivery protocol implements the Simple Mail Transfer Protocol, which defines the format of email messages, as well as the means by which those messages should be sent to mail servers. To create a custom delivery protocol component, you write code that implements the message exchange defined by some network protocol.

A delivery channel in a SQL-NS instance represents a specific delivery endpoint. The actual communication with the endpoint is handled by a delivery protocol, but the delivery channel definition captures the information required by the delivery protocol to connect to the endpoint. In other words, the delivery protocol implements a communication scheme that can be used with any endpoint that understands it, but the delivery channel identifies a specific endpoint used by the applications in a SQL-NS instance.

When you create a SQL-NS instance, you define a delivery channel in the ICF for each delivery endpoint to which notifications will be sent. A delivery channel's definition specifies a logical name (used later to identify the delivery channel), the delivery protocol used to communicate with the endpoint, and a set of arguments. The arguments typically specify the connection information needed to establish communication with the endpoint that the delivery channel represents. For example, the arguments might specify a particular server address and port. At runtime, the delivery channel arguments are passed to the delivery protocol component.

Note

Although a delivery channel is really nothing more than a named configuration representing a delivery endpoint, the concept of a delivery channel is often extended to encompass the endpoint itself. For example, it's common in SQL-NS parlance to refer to "using a delivery channel" to send a notification. What's really being used isn't the delivery channel, but rather the endpoint that the channel represents. However, this usage is generally accepted.

Another variation on this usage is to say that a delivery channel "uses" a delivery protocol. Actually, the distributor uses the delivery protocol to communicate with the endpoint that the delivery channel represents, but this usage is an accepted shorthand.

To clarify the distinction between delivery protocols and delivery channels, it may help to go back to the letter and envelope analogy used in the previous section. In this analogy, the delivery protocol is a component that implements the format and conventions of postal delivery: Letters go in envelopes, envelopes specify destination and return addresses, addresses contain postal codes, and envelopes need stamps of appropriate value. The delivery channel, on the other hand, represents a specific post office at which the delivery protocol can drop off the envelopes to have them delivered to their final recipients.

Consider a more concrete example in the context of SQL-NS. To send notifications via email using the SMTP delivery protocol, you need an SMTP server that will receive the messages from your SQL-NS application and route them to their destinations. That server is considered a delivery endpoint, and you configure your SQL-NS instance to use it by creating a delivery channel definition in the ICF. In this example, the delivery channel arguments would specify the SMTP server name and possibly other configuration options, such as authentication information. At runtime, the SMTP delivery protocol uses this information to connect to the SMTP server to deliver notifications. The nature of the configuration information required in a delivery channel definition varies, depending on the delivery protocol it uses.

Figure 10.2 illustrates the relationship between delivery protocols, delivery channels, and delivery endpoints.

Figure 10.2. Delivery protocols and delivery channels.

Notice that different delivery channels can refer to the same delivery protocol. In the SMTP example, if you have more than one SMTP server that you use to deliver email, you create a separate delivery channel definition for each one. All these delivery channels refer to the same SMTP delivery protocol.

Note

Because of the confusion around delivery protocols and delivery channels, you will often hear people talk about "implementing a custom delivery channel." This is an incorrect use of the terminology. When people say this, they actually are talking about implementing a custom delivery protocol. You cannot implement a custom delivery channel, but you can define a delivery channel that uses a custom delivery protocol to communicate with a delivery endpoint.

Routing Notifications to Delivery Channels

When subscribers register with your SQL-NS instance, they create one or more subscriber device records to represent the devices on which they want to receive notifications. Each subscriber device record specifies a delivery channel name, which identifies the delivery channel used to send notifications to the device.

When creating a subscriber device record, the delivery channel name is usually assigned by the SMI based on choices that the user makes. For example, your SQL-NS instance may use one SMTP server to deliver email to subscribers in North America, and another to deliver email to subscribers in Europe. Each SMTP server would be represented by a delivery channel with a unique name. A subscriber registering with your SMI would specify his location, and the SMI code would choose the appropriate delivery channel name when creating the subscriber device record.

When match rules insert rows into the notifications view, the values given for the DeviceName column specify the name of the subscriber device to which each generated notification should be delivered. In the early versions of the music store application prototype, we passed a hard-coded value for the device name. In Chapter 9, we changed the match rules to pass the device name specified in a subscription field. The insert trigger on the notifications view writes the given device name to the notifications table, along with the other notification data.

When the distributor reads a notification from the notifications table, it uses the subscriber ID and device name to look up the corresponding subscriber device record. It then uses the delivery channel name in this record to identify the delivery channel that should be used for delivery. After formatting the notification data, the distributor passes it to a delivery protocol component configured to use the specified delivery channel. Thus, the subscriber device represents the link between notifications and delivery channels, as shown in Figure 10.3.

Figure 10.3. Subscriber devices relate notifications and delivery channels.

[View full size image]

Note

Because subscriber device names are unique only within the context of a subscriber, it is really the combination of subscriber ID and device name that uniquely identifies a subscriber device in the system. The notifications table tracks both the subscriber ID and the device name for each notification.

Declaring Delivery Channels and Custom Delivery Protocols in the ICF

To use a delivery protocol, you must define one or more delivery channels that reference it. Delivery channels are declared in the <DeliveryChannels> element of the ICF. When declaring a delivery channel, you provide the name of the delivery protocol it uses in the <ProtocolName> element. The names of the built-in delivery protocols, File and SMTP, are well known, and you can use them directly. For example, Listing 10.1 shows the declaration of a delivery channel that uses the File delivery protocol (taken from the music store ICF).

Listing 10.1. Declaring a Delivery Channel That Uses the File Delivery Protocol in the ICF

<NotificationServicesInstance>
  ...
  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      <ProtocolName>File</ProtocolName>
      <Arguments>
        <Argument>
          <Name>FileName</Name>
          <Value>%_InstanceBaseDirectoryPath_%\FileNotifications.txt</Value>
        </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>
  ...
</NotificationServicesInstance>

Unlike the built-in delivery protocols, all custom delivery protocols must be explicitly declared before you can refer to them in delivery channel definitions. The declarations of custom delivery protocols are made in the <Protocols> element of the ICF. A custom delivery protocol declaration assigns a name to the protocol and provides the configuration information required to load the custom delivery protocol class at runtime. After declaring a custom delivery protocol, you can reference it by name in a delivery channel definition. Listing 10.2 shows the outline of a custom delivery protocol declaration and a delivery channel definition that references it. Listing 10.2 is just meant to give you an idea of the structure and placement of the ICF elements; we examine their syntax and subelements (some of which are not shown here) in detail later in this chapter.

Listing 10.2. Declaration of a Custom Delivery Protocol and a Delivery Channel That Uses It

<NotificationServicesInstance>
  ...
  <Protocols>
    <Protocol>
      <ProtocolName>MQProtocol</ProtocolName>
      ...
    </Protocol>
  </Protocols>
  <DeliveryChannels>
    ...
    <DeliveryChannel>
      <DeliveryChannelName>MQChannel</DeliveryChannelName>
      <ProtocolName>MQProtocol</ProtocolName>
      <Arguments>
        ...
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>
  ...
</NotificationServicesInstance>

Note

Don't confuse the declaration of a custom delivery protocol with the declaration of a delivery channel that uses it. The delivery protocol declaration establishes the protocol name and tells the distributor how to load it. After you've declared a custom delivery protocol, you can declare one or more delivery channels that use it.

Because delivery channels and custom delivery protocols are declared in the ICF, their declarations apply to all applications in the instance. The distributors in all the applications can load the custom delivery protocols and use them with the declared delivery channels.

Declaring Protocol Support in the ADF

When you declare a notification class in the ADF, you specify which protocols can be used to deliver notifications of that notification class. When we were using only the File delivery protocol, the NewSong notification class in the music store ADF contained a <Protocols> declaration, as shown in Listing 10.3.

Listing 10.3. Declaring Support for the File Delivery Protocol in the NewSong Notification Class

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
     <Protocols>
       <Protocol>
         <ProtocolName>File</ProtocolName>
       </Protocol>
      </Protocols>
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

The <Protocol> declaration for the File delivery protocol just specifies the protocol name. This is sufficient to allow the distributor to deliver NewSong notifications via any delivery channel that uses the File protocol. As we add support for additional protocols, we provide additional <Protocol> declarations in the notification class. The information in these declarations enables the distributor to use the specified delivery protocols to deliver notifications of the notification class.

Note

The notification class declaration is independent of the delivery channels configured for the instance. The notification class declaration just specifies the supported delivery protocols. Any delivery channel using one of the supported delivery protocols can be used to deliver notifications belonging to the notification class.

Each <Protocol> declaration specifies a protocol name, which can be the name of a builtin delivery protocol or the name of a custom delivery protocol declared in the ICF. A <Protocol> declaration can also provide protocol-specific data to be passed to the protocol component when the distributor invokes it, and a number of execution settings that control the protocol's runtime behavior. The section "Declaring Support for the SMTP Delivery Protocol in the Notification Class" (p. 369) describes these aspects of protocol declarations in detail.

Note

Although the XML element name, <Protocol>, is used in both the ICF and the ADF, the purposes of the element in each file are quite different. A protocol declaration in the ICF registers a custom delivery protocol and tells SQL-NS how to load it at runtime. In the ADF, a protocol declaration in a notification class specifies that notifications of that class can be delivered with the specified protocol. The protocol declaration may also customize the protocol's behavior for the particular notification class by supplying protocol-specific data fields and execution settings. You need to provide <Protocol> declarations only for custom delivery protocols in the ICF, but in the notification classes declared in the ADF, you need to provide <Protocol> declarations for all supported protocols, built-in or custom.

Notification Batches and Distributor Work Items

Chapter 8, "Event Providers," describes how events are grouped into units called event batches. Similarly, notifications are grouped into notification batches. By default, the generator groups all the notifications generated by a single invocation of a match rule into one notification batch. However, you can control notification batch sizes through settings in the ADF. If you specify a small notification batch size, the generator may divide the notifications generated from a single match rule invocation into several notification batches. Chapter 12, "Performance Tuning," describes the notification batch size settings in detail.

As you saw in Chapter 5, "Designing and Prototyping an Application," the SQL-NS compiler creates a notifications table for each notification class declared in the ADF. It also creates a notification batches table for each notification class, named NS<NotificationClass>NotificationBatches1, where <NotificationClass> is the name of the notification class. For example, the notification batches table for the NewSong notification class is called NSNewSongNotificationBatches1 (note the 1 at the end of the table name). This table tracks information about each notification batch, including the batch ID, the number of notifications it contains, and the times at which generation started and completed. The generator creates a row in the notification batches table for each notification batch it produces.

When the generator finishes generating a notification batch, it marks the batch record with a status code indicating that the batch is ready for distribution. Whenever it has resources to do new work, the distributor looks for new notification batches marked with this status code and begins processing them.

In the first stage of processing a notification batch, the distributor determines the delivery channel for each notification in it by looking up the corresponding subscriber device record (using the device name specified in the notification data). It then divides the batch into smaller groups of notifications based on delivery channel. Each of these smaller groups of notifications is called a distributor work item because it represents an independent unit of work for the distributor. Each distributor work item consists of the subset of notifications in the batch going to one particular delivery channel. Figure 10.4 shows the division of a notification batch into distributor work items.

Figure 10.4. Notification batches are divided into distributor work items.

[View full size image]

In Figure 10.4, Device1 and Device3 are configured to Delivery Channel 1, and Device2 and Device4 are configured to Delivery Channel 2. This configuration of subscriber devices to delivery channels would be specified in the subscriber device records, which are not shown in Figure 10.4. The four notifications in the batch illustrated are each targeted to one of the four devices. The batch is divided into two groups of notifications, based on the delivery channels of those target devices. The first distributor work item contains all the notifications in the batch going to Delivery Channel 1, and the second distributor work item contains all the notifications going to Delivery Channel 2.

Each notification class supports a particular set of protocols, as declared in its <Protocols> element. A notification batch is always divided into as many work items as there are delivery channels that use the protocols supported by its notification class.

Distributor work items are considered to be independent from one another, so the distributor attempts to process them in parallel. If an application has multiple distributors, the work items created from a single batch may be processed by different distributors.

Delivery Protocol Operation

On startup, the distributor creates a delivery protocol object for each delivery channel in the SQL-NS instance. The type of delivery protocol object created for a delivery channel depends on the protocol name specified in the delivery channel declaration. For example, the distributor creates an object of the SMTP delivery protocol class for a delivery channel that specifies SMTP as its protocol name. Similarly, if a delivery channel specifies a custom delivery protocol's name in its declaration, the distributor loads and creates an instance of the specified custom delivery protocol class for that delivery channel.

The distributor initializes each delivery protocol object by passing it the arguments specified in the associated delivery channel's declaration. Usually, these arguments provide the delivery protocol with the information it needs to connect to the endpoint that the channel represents.

As the application runs, the generator produces batches of notifications, and the distributor picks these up when they are ready. The distributor divides the batches into work items and begins processing them. During the processing of a work item, the distributor reads the notification data, passes it to the content formatter, and then passes the formatter output to one of the delivery protocol objects it created for the delivery channels in the instance. It chooses the delivery protocol object associated with the target delivery channel of the notifications in the work item. Along with the formatted notification body, the distributor also provides the delivery protocol with information about the intended recipient of each notification, including the subscriber ID and device address.

The delivery protocol may also need additional data to deliver a notification. Depending on the network protocol used, the delivery protocol may have to package the notification body into a larger message structure that includes headers required by the delivery system. For example, SMTP defines a set of message headers that specify information, such as the email subject line and encoding format. The SMTP delivery protocol needs to construct these headers for each notification it delivers. Some of the message header values are determined by the delivery protocol itself, but for others, you may need to specify the value, based on the notification data. A good example is the subject line in SMTP email messages. Because you define the notification content, you probably want to specify the text that the SMTP protocol puts in the subject line header when delivering your notifications.

SQL-NS allows you to pass additional data (beyond the formatted notification body) to a delivery protocol by defining protocol fields. Protocol fields are defined in the <Protocol> element in the notification class declaration (in the ADF) by providing a set of field names and expressions that compute their values. Protocol fields work much like computed fields (described in Chapter 9). Whereas computed fields are passed to the content formatter, protocol fields are passed to the delivery protocol. Protocol field expressions can operate over the notification data and call SQL functions.

Usually, each protocol requires a specific set of protocol fields from which it constructs message headers. The list of fields required by a protocol isn't actually declared anywhere in the ADF or ICF, but it is usually described in the documentation that accompanies the protocol. The File delivery protocol does not require any protocol fields, so none were specified in its <Protocol> declaration. The SMTP delivery protocol requires protocol fields that specify values for the various headers in an email message (including the email subject, priority, body format, and the "to" and "from" lines). When we declare support for the SMTP delivery protocol later in this chapter, you will see the required protocol fields defined.

The distributor evaluates protocol field expressions when it reads the notification data and passes the resulting values to the delivery protocol (along with the formatted notification body and the recipient information). The delivery protocol uses all this data to form the final notification message, including any message headers, and performs the operations needed to deliver the notification. It then reports the delivery status (succeeded or failed) back to the distributor by means of a callback. The distributor records this status in the notifications table.

Digest and Multicast Delivery

When digest delivery is used, it has no impact on the delivery protocol. Digesting is purely a content formatting function: The content formatter receives multiple sets of notification data and aggregates them into a single message. This single message is then passed to the delivery protocol. The delivery protocol handles it the same way it would a formatted message produced from a single notification.

Multicast delivery, on the other hand, does impact the delivery protocol. When multicast delivery is used, the distributor may pass the delivery protocol multiple sets of recipient information along with a single formatted notification body. The delivery protocol must then deliver the notification to each of the recipients. The mechanism the delivery protocol uses to do the actual delivery is not stipulated by SQL-NS. If the network protocol provides a way to send a single message to multiple recipients, the delivery protocol can use this facility. If the network protocol does not support this kind of delivery, the delivery protocol can just send a separate message to each recipient in the supplied list. Even if the delivery protocol ultimately does send a separate message to each recipient, using multicast still provides worthwhile performance gain because the notification would have been formatted only once.

Note

Multicast delivery is available only on the Enterprise Edition of SQL-NS.

Delivery Failure: Retry and Notification Expiration

Notification delivery can fail for a number of reasons. The network might be down, the delivery protocol might encounter an error, or the target address might be invalid. When a delivery failure occurs, an application may need to attempt the delivery again at a later time. SQL-NS supports retry of failed deliveries, using a configurable retry pattern.

In the protocol declarations in the ADF, you can specify retry schedules for notifications that fail to be delivered. Listing 10.10, later in this chapter, shows an example of a retry schedule declaration. A retry schedule consists of a set of delay intervals. If one or more delivery failures occur during the processing of a work item, after the work item processing completes, the distributor waits the amount of time specified in the first delay interval and then attempts to process the work item again. During the retry attempt, the distributor attempts to deliver the notifications that were not successfully delivered before. If there are still failures during the second attempt, the distributor waits the amount of time specified in the next retry delay interval and then tries to process the work item once more.

Note

All retry scheduling applies to distributor work items, not to individual notifications. The retry delays specified in the ADF govern the time the distributor waits before beginning to reprocess a work item if its previous attempt encountered failures. The delays do not necessarily translate to the exact times between successive attempts on individual notifications.

The distributor continues retrying the notification delivery until the delivery succeeds, the retry schedule is exhausted (meaning no more delay intervals in the specified list), or the notification expires. When you define a notification class, you can optionally specify an expiration age for notifications of that class using the <ExpirationAge> element. Undelivered notifications older than the expiration age are discarded without being delivered. By default, notifications have no expiration age (so they never expire). The <ExpirationAge> element is used in the notification class declaration shown in Listing 10.10, later in this chapter.

Notification expiration provides a convenient way for you to set a time limit on the lifetime of notifications. Often, the value of a notification to a subscriber diminishes over time. Notifications are supposed to provide timely information, and if they get too old, the information may not be useful to the subscriber anymore. The particular expiration ages that make sense in your applications depend on the meaning of the notification content. By setting an appropriate expiration age, you avoid unnecessary delivery attempts after notifications grow too old to be of value.

If the distributor encounters several consecutive delivery failures while trying to deliver the notifications in a work item, it abandons the whole work item, possibly leaving several notifications undelivered. The reasoning behind this behavior is that consecutive failures usually indicate that the network is down, and flooding it with additional delivery attempts may only make the problem worse. When the distributor abandons a work item because of several consecutive delivery failures, it attempts to process the work item again according to the defined retry schedule. As always, during a retry, the distributor attempts to deliver only those notifications that failed on the previous attempt or that were not attempted at all.

By default, 20 consecutive delivery failures must occur before the distributor abandons a work item. You can control this setting on a per-protocol basis by means of the <FailuresBeforeAbort> element in the ADF (described in the SQL-NS Books Online).

The Notification Distribution Views

The SQL-NS compiler creates views in the application schema that you can query for information about the delivery status of notifications. For each notification class, the compiler creates a view called NS<NotificationClass>NotificationDistribution, where <NotificationClass> is the name of the notification class. For example, the distribution view for the NewSong notification class is called NSNewSongNotificationDistribution.

This view contains a row for each delivery attempt made for each notification. The information in the view includes the notification ID, the name of the delivery channel used, the distributor that processed the notification, the delivery status, and the values of the notification fields. The SQL-NS Books Online provides detailed documentation on the columns in the notification distribution views: consult the "NS<NotificationClassName> NotificationDistribution View" topic in the SQL-NS Books Online for more information.

Caution

The writing of the logging information you see in the distribution views can significantly reduce distributor performance. SQL-NS offers a number of settings that enable or disable the logging of some optional information. By default, these settings are configured to enable logging of all available information. By selectively disabling the logging of optional information, you can reduce the logging overhead and thereby allow the distributor to run faster. In high volume applications, you may need to disable all optional distributor logging to realize the required level of performance. For more information on setting the distributor logging options, see the section "Distributor Logging Options" (p. 445) in Chapter 12.

Note

Because the notification distribution view shows one row for each delivery attempt on a notification, there can be multiple rows with the same notification ID.

Encrypting Delivery Channel Arguments

Delivery channel arguments often contain sensitive information such as server addresses and authentication credentials. For this reason, it's highly recommended that you use the argument encryption feature of SQL-NS to encrypt delivery channel arguments.

As described in Chapter 8, argument encryption is enabled in the ICF and applies to all argument values (including delivery channel arguments). See the section "Argument Encryption" (p. 237) in Chapter 8 for information on how to enable argument encryption. Since Chapter 8, the source files we've been using have enabled argument encryption. We continue this practice in this chapter.

Database Roles for the Distributor

Database objects in the instance and application schemas related to notification distribution are set up so that only members of the NSDistributor role can access them. The distributor runs in the SQL-NS engine, under the credentials of the engine's account. For it to access the required parts of the database, the account with which the engine connects to the database needs to be granted membership in the NSDistributor role. If this account is a member of the NSRunService role, it already has the required permissions because NSRunService is a member of NSDistributor.

Note

If you're using Windows Authentication, the distributor connects to the databases using the credentials of the engine's Windows account. If you're using SQL Server Authentication, the distributor connects using the SQL username and password you supplied when registering the instance.


Preparing to Work with Delivery Protocols

In this chapter, we will continue to develop the music store application by adding support for real delivery protocols. First, we add the code required to use the built-in SMTP delivery protocol; then we build a custom delivery protocol and modify the application to use it.

To work with the sample code in this chapter, you must re-create the instance and make a few other changes to your system. This section describes the steps you need to take. Make sure that you complete these steps before proceeding with the code shown in later sections of this chapter.

Build Supporting Components

We will continue to use the AddSongs program (developed in Chapter 8) to submit events into the application. If you've been following the steps in the previous chapters, you probably have this program built and ready to run, in which case you're ready to submit events in this chapter. If this is not the case, refer to the section "The AddSongs Program" (p. 243) in Chapter 8 for instructions on how to build and run AddSongs. Complete these instructions and make sure that you can start AddSongs before proceeding.

Note

If the music store database does not exist on your system (as will be the case if you performed the cleanup instructions described at the end of Chapter 9), AddSongs will display an error message if you try to connect to your SQL Server. The next section provides instructions on how to re-create the music store database, as well as the instance and application databases. After completing those instructions, you will be able to connect to the music store database with AddSongs.

Because AddSongs is our event provider, check the Submit Events for Songs Added box whenever you use AddSongs in this chapter.

In this chapter we continue using the custom content formatter developed in the previous chapter to format notifications. In Chapter 9, you built the content formatter source code that produced the content formatter assembly (NewSongNotificationFormatter.dll). Verify that this assembly exists (in the C:\SQL-NS\Samples\MusicStore\SongAlerts\CustomComponents\NewSongNotificationFormatter\bin\Debug directory). If the assembly doesn't exist in that location, you will need to build the source code to create it. Refer to the instructions in the "Implementing the Content Formatter Interface" section in Chapter 9 for a description of how to open the Visual Studio solution containing the content formatter code. Build the solution to produce the content formatter assembly.

Re-creating the SQL-NS Instance

Supplementary files containing the ICF and ADF code shown in this chapter are located in the C:\SQL-NS\Chapters\10\SupplementaryFiles directory:

  • InstanceConfiguration-2.xml adds the declaration of a delivery channel that uses the SMTP delivery protocol to what was previously in the ICF.

  • InstanceConfiguration-3.xml adds the declaration of the custom delivery protocol we develop in this chapter, as well as a delivery channel that uses it.

  • ApplicationDefinition-14.xml contains the changes to the ADF required to work with the SMTP protocol.

  • ApplicationDefinition-15.xml contains the changes to the ADF required to work with the custom delivery protocol we develop in this chapter.

The supplementary files InstanceConfiguration-2.xml and ApplicationDefinition-14.xml contain the initial ICF and ADF code from which we re-create the music store sample in this chapter. Use the following instructions to prepare the source files and get the new instance running:

1.
If you have not already done so, clean up any old version of the music store instance left on your system, as described in the "Cleanup: Preparing for the Next Chapter" sidebar (p. 346) in Chapter 9.

2.
Copy the supplementary file C:\SQL-NS\Chapters\10\SupplementaryFiles\InstanceConfiguration-2.xml over the music store instance's ICF, C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml, using the following command:

[View full width]

copy /y C:\SQL-NS\Chapters\10\SupplementaryFiles\InstanceConfiguration-2.xml C:\SQL-NS
  \Samples\MusicStore\InstanceConfiguration.xml

3.
Copy the supplementary file C:\SQL-NS\Chapters\10\SupplementaryFiles\ApplicationDefinition-14.xml over the music store application's ADF, C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml, using the following command:

[View full width]

copy /y C:\SQL-NS\Chapters\10\SupplementaryFiles\ApplicationDefinition-14.xml C:\SQL-NS
  \Samples\MusicStore\SongAlerts\ApplicationDefinition.xml

4.
From a Notification Services Command Prompt on your development machine, navigate to the music store instance's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

5.
Run create_music_store_database.cmd. This rebuilds the music store database.

6.
Run create_with_argument_key.cmd to compile the instance and application from scratch.

7.
Run register_with_argument_key.cmd to register the instance.

8.
Run grant_permissions.cmd to assign the appropriate database permissions.

9.
Run enable.cmd to enable the instance.

Caution

Do not start the music store instance's Window service (NS$MusicStore) at this time. As instructed in the following section, "Change Service Account Permissions" (p. 363), you will need to change the permissions of the service account. If the service is running at the time you do this, you'll be required to stop and restart it.

Change Service Account Permissions

When you test the music store application with the SMTP delivery protocol later in this chapter, you'll use a local SMTP server installed on your development machine as the delivery endpoint. The "Installing the SMTP Service" section (p. 366) describes how to install the SMTP service on your development machine in preparation for this testing.

Because of the way the .NET Framework Mail APIs (used in the implementation of the built-in SMTP protocol) interact with Internet Information Services (of which the SMTP service is a part), sending email via the local SMTP server requires Administrator privileges. However, from a security perspective, it's always best to run the SQL-NS engine under a low-privileged account; to test the capabilities of the SMTP delivery protocol in this chapter, you need to temporarily grant Administrator privileges to the engine's account (the account used to run the SQL-NS Windows service). The "Restoring Your System" section (p. 389) at the end of this chapter provides instructions on how to revoke these privileges after you're finished working with SMTP in this chapter.

Note that if you're using the SMTP delivery protocol with a remote SMTP server (as is most often the case in production environments), you do not need to grant Administrator privileges to the service account. This step is necessary only in the development environment we're using.

Use the following instructions to add the service account to the Administrators group on your development machine:

Note

To complete these instructions, you will need to know the name of the service account you created in Chapter 2, "Getting Set Up." If you don't remember the account name, open the C:\SQL-NS\Common\setenv.cmd file and look at the value you specified for the NS_SERVICE_USER_NAME environment variable.

1. | From the Start menu, right-click My Computer and choose Manage from the context menu that appears. The Computer Management window opens.
2. | In the tree in the left pane of the Computer Management window, select the Local Users and Groups node.
3. | Double-click the Groups folder in the right pane.
4. | Double-click the Administrators group to open its properties page.
5. | Click the Add button beneath the list of group members.
6. | In the Select Users, Computers, or Groups dialog box that appears, type the name of the service account. If your service account is a domain account, make sure that you specify the domain name, using the DOMAIN\User syntax (for example, PATHERNET\SQL-NS_Service).
7. | Click OK to close the dialog box and return to the Administrators group properties page. The service account should now appear in the list of members.
8. | Click OK to close the Administrators group property page and then close the "Computer Management" window.

Adding Subscriber Devices for New Delivery Channels

In Chapter 9, we used a T-SQL script to add a set of hard-coded subscribers, subscriber devices, and subscriptions to the instance and application. The data specified various device types and locales that allowed us to test the content formatting capabilities of the application. However, all the subscriber devices specified the delivery channel name as FileChannel, the name of the delivery channel that uses the File delivery protocol.

In this chapter, we create two new delivery channels: one that uses the SMTP delivery protocol, and another that uses the custom delivery protocol that we build. To test the new delivery protocols with the music store application, we need subscriber devices that refer to the new delivery channels. Again, we'll use T-SQL scripts to enter the required data.

The C:\SQL-NS\Chapters\10\Scripts directory contains three T-SQL scripts:

  • AddSubscribers.sql adds a set of subscriber records.

  • AddSMTPSubscriberDevicesAndSubscriptions.sql adds subscriber devices configured to the delivery channel that uses SMTP and a set of subscriptions that reference these devices.

  • AddMQSubscriberDevicesAndSubscriptions.sql adds subscriber devices configured to the delivery channel that uses our custom delivery protocol and a set of subscriptions that reference these devices.

In these scripts, the subscriber devices configured to the SMTP delivery channel are of the Email device type. Those configured to the delivery channel that use the custom delivery protocol are of the TextMessageDevice type.

The delivery channel that uses SMTP is defined in the ICF you already compiled in the "Re-creating the SQL-NS Instance" section (p. 362) earlier. The delivery channel that uses the custom delivery protocol will be added to the instance when we build the protocol itself. Both delivery channel definitions are described in detail later in this chapter. Because SQL-NS does not allow subscriber devices to reference nonexistent delivery channels and only the SMTP delivery channel is currently defined, we can add only the Email devices at this time. We cannot run the third script in the preceding list, AddMQSubscriberDevicesAndSubscriptions.sql, until we add the second delivery channel, later on.

For now, complete the following steps to submit the subscriber records, the subscriber device records for the SMTP delivery channel, and the corresponding subscription records:

1. | Open AddSubscribers.sql and AddSMTPSubscriberDevicesAndSubscriptions.sql (from C:\SQL-NS\Chapters\10\Scripts) in Management Studio.
2. | Take a look at the data hard-coded in these files so that you're familiar with the subscriber, subscriber device, and subscription records that will be submitted.
3. | Execute AddSubscribers.sql first, then execute AddSMTPSubscriberDevicesAndSubscriptions.sql.

Installing the SMTP Service

To test the application with the SMTP delivery protocol, you need to have the SMTP Service installed on your development machine. However, the service need not be running (and, as described later in this chapter, should not be running while you're testing the application).

Use the following instructions to install the SMTP Service:

1. | From your Start menu, open Control Panel and then Add or Remove Programs.
2. | In the left pane of the Add or Remove Programs window, choose the Add or Remove Windows Components icon. This opens the Windows Components Wizard.
3. | Select the Internet Information Services (IIS) item in the components list. In some versions of Windows, Internet Information Services (IIS) appears under the Application Server item in the components list. If you don't see Internet Information Services (IIS) in the first list of components, select Application Server, then click Details. Internet Information Services (IIS) should be in the subcomponents list that appears.
4. | With Internet Information Services (IIS) selected, click the Details button to open the IIS subcomponents dialog box.
5. | Find the SMTP Service item in the subcomponents list and check the box next to it. If the box is already checked, you already have the SMTP Service installed, so you can cancel the wizard and skip the rest of these instructions.
6. | Click OK to close the subcomponents dialog box and then click Next in the wizard. Windows Setup then installs the necessary files (you may be prompted to insert your Windows installation CDs).
7. | Finish the wizard and close Add or Remove Programs, as well as the main Control Panel window.

Message Queuing for the Custom Delivery Protocol

The custom delivery protocol that we build in this chapter uses Message Queuing to deliver notifications. In Chapter 8, we also used Message Queuing to build one of the custom event providers. If you installed Message Queuing in Chapter 8, you are already set up to work with the custom delivery protocol in this chapter. If you did not install Message Queuing earlier but want to use the custom delivery protocol that we build here, refer to the "Installing Message Queuing" section (p. 245) in Chapter 8 and carry out the installation instructions given there.


Using the Built-in SMTP Delivery Protocol

In this section, we modify the music store application to support the built-in SMTP delivery protocol. Using this protocol allows the application to deliver notifications via email. Although the code in this section is specific to the music store application, the general techniques shown can be used to add SMTP support to any application.

Before continuing, it's important to take stock of where we are in the development process. We completed the prototype that showed the application's core logic and then added a subscription management interface, a real event provider, and a real content formatter. There is still work to do before the music store application is ready to be deployed in a production environment, but adding support for a real delivery protocol finally makes it seem less like a prototype and more like real application.

Creating an SMTP Delivery Channel

To use the SMTP delivery protocol, we need to declare a new delivery channel to represent an SMTP delivery endpoint. Listing 10.4 shows the delivery channel declaration in the ICF.

Listing 10.4. Declaration of an SMTP Delivery Channel

<NotificationServicesInstance>
  ...
  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      ...
    </DeliveryChannel>

    <DeliveryChannel>
      <DeliveryChannelName>SMTPChannel</DeliveryChannelName>
      <ProtocolName>SMTP</ProtocolName>
      <Arguments>
        <Argument>
          <Name>SmtpServer</Name>
          <Value></Value>
        </Argument>
       <Argument>
         <Name>BodyEncoding</Name>
         <Value>utf-8</Value>
       </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>
  ...
</NotificationServicesInstance>

The declaration appears in a new <DeliveryChannel> element within the existing <DeliveryChannels> element. Note that the new delivery channel does not replace the old file delivery channel. It's usually a good idea to leave the file delivery channel in place because it provides an easy mechanism for debugging distribution problems, should you need to later on.

The <DeliveryChannel> declaration specifies a name, SMTPChannel, by which the channel can be identified. As you may recall from looking at the script used to add subscriber devices, this is the channel name used in the creation of the Email subscriber device records. The <ProtocolName> element specifies the name of the delivery protocol that the channel uses: SMTP.

The <Arguments> element specifies the delivery channel arguments that the SQL-NS engine passes to the delivery protocol at runtime. Each delivery protocol expects a different set of arguments; the ones shown in Listing 10.4 are specific to the SMTP delivery protocol. The first argument, SmtpServer, specifies the name of the SMTP server to which the protocol should send notifications. Omitting this argument (or providing a blank value as Listing 10.4 does) causes the delivery protocol to use the local SMTP service.

Caution

Do not specify the actual local machine name in the SmtpServer argument because doing so causes the SMTP protocol to behave in a way that requires the SMTP Service to be running. As is explained later, in our testing, we leave the SMTP Service in the stopped state to prevent our test emails from being relayed off the box. Supplying a blank value for SmtpServer, or omitting the argument entirely, causes the SMTP protocol to use the local SMTP Service without requiring it to be running.

Note

Although using the local SMTP service is useful for testing the application while it is still in development, it is not a recommended practice in deployment. In a production environment, the local SMTP service usually acts as a relay to a remote SMTP server that does the delivery. Queuing to the local server typically has a negative performance impact, so it's recommended that you specify the name of the remote SMTP server in the channel arguments and let the protocol send mail to it directly.

The second delivery channel argument specifies the character encoding to use for the email messages. Table 10.1 shows the legal values for this argument, although not all these encodings are supported on every system. The example shown in Listing 10.4 uses UTF-8 encoding, but in general, you need to choose a value based on the content of your notifications.

Table 10.1. Allowed Values for the BodyEncoding Delivery Channel Argument Required by the SMTP Delivery Protocol

Encoding Name

BodyEncoding Value

ASCII

us-ascii

UTF-7

utf-7

UTF-8

utf-8

Unicode

utf-16

Big Endian Unicode

unicodeFFFE

Declaring Support for the SMTP Delivery Protocol in the Notification Class

To use the SMTP protocol to deliver NewSong notifications, we must declare support for it in the NewSong notification class. Listing 10.5 shows this declaration.

Listing 10.5. Declaration of SMTP Protocol Support in the NewSong Notification Class

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <Protocols>
        <Protocol>
          <ProtocolName>File</ProtocolName>
        </Protocol>
        <Protocol>
          <ProtocolName>SMTP</ProtocolName>
          <Fields>
            <Field>
              <FieldName>Subject</FieldName>
              <SqlExpression>
                N&apos;Music Store Song Alert&apos;
              </SqlExpression>
            </Field>
            <Field>
              <FieldName>From</FieldName>
              <SqlExpression>
                N&apos;alerts@musicstore.com&apos;
              </SqlExpression>
            </Field>
            <Field>
              <FieldName>To</FieldName>
              <SqlExpression>
                DeviceAddress
              </SqlExpression>
            </Field>
            <Field>
              <FieldName>Priority</FieldName>
              <SqlExpression>
                N&apos;Normal&apos;
              </SqlExpression>
            </Field>
            <Field>
              <FieldName>BodyFormat</FieldName>
              <SqlExpression>
                N&apos;text&apos;
              </SqlExpression>
            </Field>
          </Fields>
        </Protocol>
      </Protocols>
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

Beneath the existing <Protocol> declaration for the File delivery protocol, there is a new <Protocol> declaration for SMTP. The declaration specifies the protocol name and then defines a set of protocol fields.

The protocol fields shown in Listing 10.5 are the ones required by the SMTP delivery protocol. Other delivery protocols require different protocol fields, based on the data they need to deliver notifications. Each protocol field consists of a name and a SQL expression that the distributor evaluates to compute the field value when it reads notification data from the notifications table. When the distributor calls the delivery protocol to deliver a notification, it passes the formatted notification message, along with the values of the protocol fields. Protocol field expressions may be constants, function calls, or expressions over the notification fields. The expressions may also reference the following additional fields:

  • SubscriberId The ID of the subscriber for whom the notification was generated

  • DeviceAddress The DeviceAddress property of the subscriber device associated with the notification

  • DeviceTypeName The DeviceTypeName property of the subscriber device associated with the notification

  • SubscriberLocale The locale code specified for the notification

The declaration in Listing 10.5 defines the following SMTP protocol fields: Subject, From, To, Priority, and BodyFormat. The Subject value is used to populate the subject field of the resulting email and is specified as a constant string. Like computed fields (explained in the section "Adding Computed Fields," p. 318, in Chapter 9), the SQL expressions that define protocol field values must be written so that they could appear in the SELECT clause of a SQL query. You can use constant strings, but they must be surrounded with single quotes (recall that SQL uses single, not double, quotes for literal strings). Because the single quote character is a reserved character in XML, we have to use the &apos; escape sequence (entity reference) in its place. The SQL expression for the Subject field is given as N&apos;Music Store Song Alert&apos;, which is really N'Music Store Song Alert' when the escape sequences are replaced. The N character at the start of the string specifies that it is a Unicode string. When the distributor evaluates the expression for the Subject field, the resulting value will be the string Music Store Song Alert.

Caution

A common mistake when declaring protocol field expressions is to specify string constants without quotes. This leads to a compile error because unquoted string constants cannot be used in a SQL query.

The From and To fields provide values for the sender and recipient addresses on the email message. The From field is defined with a constant string expression, much like the Subject field. Again, the &apos; escape sequence is used in place of the single quote character. The expression for the To field is given as DeviceAddress. This means that the value of this field will be taken from the DeviceAddress property of the subscriber device record for the notification. The T-SQL script that created the subscriber device records in the music store instance specified email addresses for the DeviceAddress property on email devices. In general, the DeviceAddress property on a subscriber device can be any string that the delivery protocol can interpret.

The Priority and BodyFormat protocol fields in Listing 10.5 are both constant strings. The valid values for Priority may be different on the particular email server you are using, but on Microsoft Exchange Server, Low, Normal, and High are valid values. The BodyFormat field indicates whether the notification body (produced by the content formatter) contains text or HTML. The valid values for this field are text and html.

Testing SMTP Delivery

The code in Listings 10.4 and 10.5 is already in the ICF and ADF files you compiled when you set up the instance in the "Re-creating the SQL-NS Instance" section (p. 362) earlier in this chapter. Assuming that you completed the steps in that section, all you need to do to test SMTP support is submit some events that match the email subscriptions. (Make sure that you have also completed the instructions in the "Adding Subscriber Devices for New Delivery Channels" section, p. 365, before proceeding.)

Before running the test, we will stop the SMTP service. As a result, when the SMTP delivery protocol delivers the notifications, they will end up in the local SMTP queue, but they will not actually be sent out. We can verify that the application is working by looking at the messages in the SMTP queue. You should always have the SMTP service stopped when running tests on notification applications that use the SMTP delivery protocol because it prevents the test notifications from being delivered off your machine and actually reaching users.

Note

If other services or applications on your development machine use the local SMTP service, those applications cannot send email while the service is stopped. The "Restoring Your System" section (p. 389) at the end of this chapter describes how to restart the SMTP service when you're finished testing.

Use the following instructions to stop the SMTP service, submit the events, and observe the email notifications:

1.
Stop the local SMTP service by issuing the following command at a command prompt:

net stop SMTPSvc

2.
Start the SQL-NS service for the music store instance. If it is was running at the time you completed the instructions in the "Change Service Account Permissions" section (p. 363), stop and restart it. Use the following command to start the service:

net start NS$MusicStore

3.
Open the AddSongs program, located under the C:\SQL-NS\Samples\MusicStore\SongAlerts\AddSongs\bin\Debug directory, and connect to your music store database. If you had the program open from the previous chapter, before you deleted and re-created the music store database, close and re-open it now.

4.
Add the following song data into the AddSongs program:

Album Title: The Look of Love
Artist Name: Diana Krall
Genre: Jazz
Song 1: I Remember You
Song 2: Cry Me a River

5.
Check the Submit Events for Songs Added box and click the Add to Database button.

6.
In Windows Explorer, open the Pickup directory in your SMTP service's mailroot directory. On most systems, this directory is located at C:\Inetpub\mailroot\Pickup. When the application delivers notifications, email files will appear in this directory.

7.
About 30 seconds after you submitted the events, you should see two email files appear in the Pickup directory window. Open these files to examine their contents.

Figure 10.5 shows an email message generated by the music store application. The messages on your system should look similar. Notice that the body of the email is the text generated by our custom content formatter (one of the messages should be formatted in French).

Figure 10.5. A notification email generated by the music store application.

[View full size image]

The email notifications generated are real, and if the SMTP service were running, it would pick them up from the Pickup directory and route them to their recipients. As it currently stands, you have a working notification application that can send email!


Building a Custom Delivery Protocol

If you want to support delivery methods other than email, you need a custom delivery protocol. This section describes how to build a custom delivery protocol and integrate it into your SQL-NS application.

As mentioned in the early part of this chapter, several software vendors produce reusable custom delivery protocols that you can leverage. If you obtain a custom delivery protocol from one of these vendors, you can use the configuration techniques described in the later parts of this section to deploy it in your SQL-NS instance and use it in your applications.

The Custom Delivery Protocol Interface

Like other custom components in SQL-NS, custom delivery protocols are written as classes that implement a specific SQL-NS interface. The interface for delivery protocols is called IDeliveryProtocol and is shown in Listing 10.6.

Listing 10.6. The IDeliveryProtocol Interface

public interface IDeliveryProtocol
{
    void Initialize(
        StringDictionary channelArgs,
        bool multicast,
        NotificationStatusCallback nsc);

    void DeliverNotification(
        NotificationHeaders[] headersList,
        string body);

    void Flush();

    void Close();
}

The interface defines the following four methods:

  • Initialize() Sets up the delivery protocol

  • DeliverNotification() Delivers a single notification (or, in the case of multicast, several notifications in a single multicast group)

  • Flush() Completes or aborts any pending delivery operations in preparation for shutdown

  • Close() Cleans up any resources used by the delivery protocol

Figure 10.6 shows the operation of the IDeliveryProtocol methods graphically.

Figure 10.6. Operation of custom delivery protocol methods.

[View full size image]

When the distributor starts processing a work item, it calls the delivery protocol's Initialize() method, passing it the following items:

  • The arguments specified in the declaration of the delivery channel to which the notifications in the work item are targeted

  • A flag indicating whether multicast delivery will be used

  • A delegate to a callback function that the protocol can use to report the delivery status of notifications it handles

In the Initialize() method, a protocol typically saves the multicast flag and the callback delegate in private member variables and then processes the delivery channel arguments. The delivery channel arguments usually contain the information needed by the protocol to connect to the delivery endpoint that the channel represents. When implementing a custom delivery protocol, you decide the arguments that it requires.

The distributor calls the DeliverNotification() method for each formatted notification that needs to be delivered. It passes two arguments to DeliverNotification():

  • An array of "notification headers" that specify information about each intended recipient of the notification

  • A string containing the formatted notification body

To understand the contents of the notification headers array, think about how notifications are represented in the notifications table. The table contains a row for each generated notification that specifies the notification data, as well as information about the intended recipient (such as the subscriber ID and device name). You can think of each row as representing a separate delivery instruction saying, "send this notification data to this subscriber on this device." When multicast delivery is used, the distributor groups notifications that have common notification data and formats that data just once. But it still has to carry out the delivery instructions represented by each individual notification row.

The notification headers array passed to the DeliverNotification() method essentially contains all the individual delivery instructions that apply to the given formatted notification message. The headers array contains a NotificationHeaders object for every notification row in a multicast group. If multicast delivery is not used, the headers array always contains just one element.

Each NotificationHeaders object in the array contains the following information:

  • A NotificationState object that identifies a notification row. (Think of this as an opaque identifier that the distributor uses to keep track of which row the NotificationHeaders object represents.)

  • A dictionary of protocol field values obtained by applying the protocol field expressions to the data in the notification row.

  • A RecipientInfo object that provides the subscriber ID and DeviceAddress property of the subscriber device referenced in the notification row.

Using the information passed in, the DeliverNotification() method attempts to deliver the notification message. Notice that the DeliverNotification() method does not return a value indicating whether the delivery succeeded or failed. Instead, delivery status is reported by means of the NotificationStatusCallback delegate (originally passed to the Initialize() method). This delegate is defined with the following signature:

public delegate void NotificationStatusCallback(params NotificationStatus[] status);

When its delivery attempts complete, the delivery protocol invokes the delegate, passing in the delivery status of one or more notifications. The delegate takes a variable number of NotificationStatus objects, each of which represents the delivery status of one notification. The delivery protocol must call the notification status callback delegate with one NotificationStatus object for each NotificationHeaders object it received in DeliverNotification(). Each NotificationStatus object has the following properties (which the delivery protocol sets appropriately):

  • NotificationState The NotificationState object that identifies the notification for which status is being reported (obtained from the NotificationHeaders object passed to the DeliverNotification() method).

  • NotificationText Specifies the text of the final notification message that the delivery protocol sent out, including any headers appended to the notification body. (This property is optional and is only useful for debugging purposes.)

  • StatusInfo Specifies additional information about the delivery status, such as an error message in the case of failure. (Again, this property is optional and is used only for debugging and diagnostic purposes.)

  • Succeeded A Boolean flag indicating whether the delivery succeeded.

  • TimeStamp A time stamp that indicates when the delivery occurred.

The delivery protocol can report the status of notifications one at a time or in sets. There is no requirement that the status of notifications be reported in order, or that the status of all the notifications passed to one DeliverNotification() invocation be reported together.

Caution

Although there are no restrictions on the grouping or ordering of notification status reports through the notification status callback, the delivery protocol must eventually report the status of every notification it handles. In other words, the total number of NotificationStatus objects passed in all the invocations of the callback must equal the total number of NotificationHeaders objects passed to the delivery protocol in all the invocations of DeliverNotification().

The use of the callback mechanism for reporting notification status allows the delivery protocol to implement asynchronous delivery. Instead of performing the delivery operation immediately when DeliverNotification() is called, the delivery protocol can queue a delivery request and return from DeliverNotification quickly. It can then service the queued delivery requests asynchronously and report their delivery status through the call-back. Delivery protocols implemented asynchronously usually perform much better than their synchronous counterparts because they do not block the distributor while waiting for a delivery to complete. As soon as DeliverNotification() returns, the distributor can start processing the next notification in the distributor work item. This can be an advantage if the delivery operations involve network latency, as they often do.

The distributor calls the delivery protocol's Flush() method at the end of a distributor work item. In the Flush() method, the protocol should either complete or abort any pending notification delivery attempts and report their status through the notification status callback. After Flush() returns, there should be no more outstanding notification requests. It is illegal for the delivery protocol to call the notification status callback after Flush() returns.

The Flush() method is usually important only if the protocol implements asynchronous delivery. In these delivery protocols, the Flush() should ensure that asynchronous delivery requests either complete or are canceled. In synchronous delivery protocols, each delivery operation completes by the end of the DeliverNotification() method, so Flush() usually doesn't do anything.

After Flush() returns and the distributor performs the cleanup it needs to do to finish processing the work item, it calls the protocol's Close() method. Close() should clean up any resources held by the protocol during its operation.

The distributor may reuse an instance of a delivery protocol class across the processing of several distributor work items. However, the distributor makes the following guarantees:

1. | Initialize() will be called at the start of a work item and will not be called again until Flush() and Close() have been called.
2. | DeliverNotification() will be called zero or more times, depending on the number of notifications in the work item.
3. | Flush() will be called at least once after Initialize().
4. | Close() will be called after the final call to Flush(). After Close() returns, no other methods will be called on the delivery protocol until Initialize() is called again.

Note

If you need to debug a custom delivery protocol, you can either attach a debugger to the SQL-NS Windows service process, or you can start the service as a console application from within a debugger. The "Debugging a Custom Component" topic in the SQL-NS Books Online provides instructions for debugging custom components that run in the SQL-NS Windows service.

Implementing the Custom Delivery Protocol

In this section, we implement the code for a custom delivery protocol that uses Message Queuing to deliver notifications to a message queue. Open the code for this custom delivery protocol using the following instructions:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\CustomComponents\MQDeliveryProtocol directory.
2. | Open the solution file MQDeliveryProtocol.sln in Visual Studio.

The delivery protocol is implemented in the MQDeliveryProtocol.cs file. Listing 10.7 shows the code in this file.

Listing 10.7. Implementation of the MQDeliveryProtocol

namespace MusicStoreCustomComponents
{
    /// <summary>
    /// Class that represents notifications sent via
    /// the MQ Delivery Protocol.
    /// </summary>
    public class MQNotification
    {
        ...

        // Public accessors for properties.
        public string MessageId
        {
            ...
        }

        public string SubscriberId
        {
            ...
        }

       public string TargetAddress
       {
           ...
       }

      public string Body
      {
           ...
      }
   }
     /// <summary>
     /// Implements a custom delivery protocol that
     /// sends notifications over MSMQ.
     /// </summary>
     class MQDeliveryProtocol : IDeliveryProtocol
     {
         private string queueName;
         private bool multicast;
         private NotificationStatusCallback nsc;
         private MessageQueue queue;

         public void Initialize(
             StringDictionary channelArgs,
             bool multicast,
             NotificationStatusCallback nsc)
         {
             // Read the queue name from the channel arguments.
             if (channelArgs.ContainsKey("QueueName") &&
                 channelArgs["QueueName"] != "")
             {
                 this.queueName = channelArgs["QueueName"];
             }
             else
             {
                 // QueueName is a required delivery channel argument,
                 // so throw an exception if it isn't specified.
                 throw new ArgumentException(
                     "A required delivery channel argument was not specified.",
                     "QueueName");
             }

             // Store the multicast parameter and notification status
             // callback delegate.
             this.multicast = multicast;
             this.nsc = nsc;

             // Establish a connection to the message queue.
             if (MessageQueue.Exists(queueName))
             {
                 // The message queue exists, so just obtain a
                 // reference to it.
                 this.queue = new MessageQueue(this.queueName);
             }
             else
             {
                 // The message queue does not exist, so create it.
                 this.queue = MessageQueue.Create(this.queueName);
             }
          }

          public void DeliverNotification(
              NotificationHeaders[] headersList,
              string body)
          {
              int numNotifications = headersList.Length;
              MQNotification[] notifications;
              NotificationStatus[] statusObjects;

              // Create arrays to hold the notification objects and
              // status objects.
              notifications = new MQNotification[numNotifications];
              statusObjects = new NotificationStatus[numNotifications];

              // For each header in the given list, create a notification
              // object and a status object.
              for (int i = 0; i < numNotifications; i++)
              {
                   NotificationHeaders headers = headersList[i];

                   notifications[i] = new MQNotification(
                       headers.ProtocolFields["MessageId"],
                       headers.RecipientInfo.SubscriberId,
                       headers.RecipientInfo.DeviceAddress,
                       body);

                   try
                   {
                       // Attempt the delivery.
                       this.queue.Send(notifications[i]);

                       // Since no exception was thrown, the delivery
                       // status is a success.
                       statusObjects[i] = new NotificationStatus(
                           headers.NotificationState,
                           true,
                           "",
                           body,
                           (object)DateTime.UtcNow);
                   }
                   catch (Exception ex)
                   {
                        // The Send() operation threw an exception, so
                        // record the notification status as failed.
                        statusObjects[i] = new NotificationStatus(
                            headers.NotificationState,
                            false,
                            ex.Message,
                            body,
                           (object)DateTime.UtcNow);
                   }
               }

               // Report all status back to the distributor.
               nsc(statusObjects);
           }

           public void Flush()
           {

               // Do nothing.
           }

           public void Close()
           {
               // Close the message queue.
               this.queue.Close();
           }
     }
}

The first part of the source file defines a class, MQNotification, that represents the structure of the messages that will be delivered to the message queue. The delivery protocol creates an instance of this class for each notification it is asked to deliver and sends this instance to the message queue.

The MQNotification class defines properties for the various aspects of a notification delivered via the MQDeliveryProtocol, including message ID, subscriber ID, target address, and notification body. This is a delivery protocol made up purely for the purposes of illustration, and these fields were chosen somewhat arbitrarily as examples of the headers that typical delivery protocols might use. Notice that the message structure is not specific to the music store application. The delivery protocol described here could be used with any application that wants to deliver to a message queue.

The delivery protocol operations are implemented in the MQDeliveryProtocol class, which implements the IDeliveryProtocol interface. The Initialize() method saves the multicast flag and the notification status callback, and then looks in the channel arguments dictionary for an argument called QueueName. This is a required argument that tells the protocol the name of the queue to connect to. If the argument is not specified, Initialize() throws an exception. Because the queue name is not hard-coded, but instead obtained dynamically from a delivery channel argument, the MQDeliveryProtocol can be used with any message queue. You can create a delivery channel for the message queue you want to use, configure it to use the MQDeliveryProtocol, and specify the queue name through the QueueName argument. This is an example of how a delivery channel is used to represent a delivery endpoint (the specific message queue). The channel arguments tell the delivery protocol how to connect to that endpoint.

After reading the channel arguments, Initialize() creates a MessageQueue object to represent the message queue. If the queue already exists, the protocol just creates a reference to it. If the queue does not exist, the protocol actually creates it.

The DeliverNotification() method determines the number of notification headers passed in and creates two arrays with this number of elements. The first array is used to store the MQNotification objects that it sends to the message queue. The second array stores the NotificationStatus objects that it will pass to the notification status callback to report the status of each notification.

For each element in the headers array, the protocol creates an MQNotification object and sets the properties using the value of a protocol field, the recipient information, and the formatted notification body. In this case, the protocol uses a protocol field called MessageId that provides a unique identifier for the message. When we look at the protocol declaration in the ADF, you will see this protocol field defined.

After creating the MQNotification object, the protocol attempts to deliver the notification by calling the Send() method on the message queue object. If this method does not throw an exception, the code assumes that the delivery succeeded and creates a notification status object indicating success. If an exception is thrown, the code catches it and creates a notification status object indicating failure. The StatusInfo property of the status object is set to the value of the exception message.

Note

All the information returned by the delivery protocol through the notification status objects can be viewed in the notification distribution views, as described in the section, "The Notification Distribution Views" (p. 360).

After the delivery protocol attempts to deliver all the notifications, it reports their status using the notification status callback. It passes the entire array of NotificationStatus objects to the callback delegate.

Because this is a synchronous delivery protocol, the Flush() method has no work to do; all delivery attempts complete before DeliverNotification() returns. The Close() method closes the message queue object and then returns. This is the only operation required to clean up the delivery protocol state.

Declaring the Custom Delivery Protocol in the ICF

To use the custom delivery protocol, we need to declare it in the ICF. Listing 10.8 shows this declaration.

Listing 10.8. Declaration of the Custom Delivery Protocol

[View full width]
<NotificationServicesInstance>
  ...
  <Protocols>
    <Protocol>
      <ProtocolName>MQProtocol</ProtocolName>
      <ClassName>MusicStoreCustomComponents.MQDeliveryProtocol</ClassName>
      <AssemblyName>%_InstanceBaseDirectoryPath_%\CustomComponents\MQDeliveryProtocol\bin
  \Debug\MQDeliveryProtocol.dll</AssemblyName>
    </Protocol>
  </Protocols>

  <DeliveryChannels>
    ...
  </DeliveryChannels>
  ...
</NotificationServicesInstance>

The declaration appears in a <Protocol> element, inside the <Protocols> element. It specifies a protocol name, which can then be used in delivery channel declarations and notification class declarations in the ADF to identify the custom delivery protocol. The declaration also specifies the class and assembly names that the distributor will use to load the custom delivery protocol at runtime. As with all other SQL-NS custom components, the class name must be provided in fully qualified form, with the namespace prefix, and the assembly name must include a full path. The assembly name given here refers to the delivery protocol assembly that will be built when we compile the delivery protocol source code, as described in the section, "Testing the Custom Delivery Protocol," (p. 386).

Creating a Delivery Channel for the Custom Delivery Protocol

After the custom delivery protocol has been declared, we can create a delivery channel that uses it, as shown in Listing 10.9.

Listing 10.9. Declaration of a Delivery Channel Using the Custom Delivery Protocol

<NotificationServicesInstance>
  ...
  <Protocols>
    <Protocol>
      <ProtocolName>MQProtocol</ProtocolName>
      ...
    </Protocol>
  </Protocols>

  <DeliveryChannels>
    <DeliveryChannel>
      <DeliveryChannelName>FileChannel</DeliveryChannelName>
      ...
    </DeliveryChannel>

    <DeliveryChannel>
      <DeliveryChannelName>SMTPChannel</DeliveryChannelName>
      ...
    </DeliveryChannel>

    <DeliveryChannel>
      <DeliveryChannelName>MQChannel</DeliveryChannelName>
      <ProtocolName>MQProtocol</ProtocolName>
      <Arguments>
        <Argument>
          <Name>QueueName</Name>
          <Value>.\Private$\NewSongNotificationQueue</Value>
        </Argument>
      </Arguments>
    </DeliveryChannel>
  </DeliveryChannels>
  ...
</NotificationServicesInstance>

The delivery channel declaration is added below the other delivery channel declarations in the <DeliveryChannels> element in the ICF. It specifies a delivery channel name and references the protocol name declared in Listing 10.8. It also defines the one required delivery channel argument, the queue name.

Supporting the Custom Delivery Protocol in the Notification Class

Finally, to deliver NewSong notifications with the custom delivery protocol, we need to add a <Protocol> declaration for it to the NewSong notification class, as shown in Listing 10.10.

Listing 10.10. Declaring Support for the Custom Delivery Protocol in the NewSong Notification Class

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      ...
      <Protocols>
        <Protocol>
          <ProtocolName>File</ProtocolName>
        </Protocol>
        <Protocol>
          <ProtocolName>SMTP</ProtocolName>
          ...
        </Protocol>
        <Protocol>
          <ProtocolName>MQProtocol</ProtocolName>
          <Fields>
            <Field>
              <FieldName>MessageId</FieldName>
              <SqlExpression>
                NEWID()
              </SqlExpression>
            </Field>
          </Fields>
          <ProtocolExecutionSettings>
            <RetrySchedule>
              <RetryDelay>P0DT00H05M00S</RetryDelay>
              <RetryDelay>P0DT00H15M00S</RetryDelay>
            </RetrySchedule>
          </ProtocolExecutionSettings>
        </Protocol>
      </Protocols>
      <ExpirationAge>PT06H00M00S</ExpirationAge>
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

The new <Protocol> declaration appears after the other protocol declarations, which remain valid. This means that notifications of this notification class can be delivered over multiple delivery protocols.

The protocol declaration references the protocol name and declares the one protocol field that the protocol requires. This protocol field provides a message ID by calling the NEWID() SQL function to generate a new identifier. You saw the value of this protocol field accessed in the implementation of the delivery protocol's DeliverNotification() method, in Listing 10.7.

For illustration purposes, Listing 10.10 also shows how a protocol declaration can specify a retry schedule for failed notifications. In this case, the retry schedule specifies two delay intervals: 5 minutes and 15 minutes. If one or more delivery failures occur in the processing of a work item when using the MQProtocol, the distributor will wait 5 minutes and then process the work item again, attempting to deliver those notifications that were not successfully delivered before. If there are still failures, the distributor will try once more after waiting 15 minutes. Because the retry schedule is specified within the <Protocol> declaration, you can specify different retry patterns for different protocols.

Note

The delays in a retry schedule are relative to one another, not relative to the first processing attempt on the work item. In the example shown in Listing 10.10, the first retry attempt will happen 5 minutes after the original attempt, and the second retry attempt will happen 15 minutes after the first retry attempt (20 minutes after the original attempt).

Also for illustration, Listing 10.10 shows how the <ExpirationAge> element can be used to specify an expiration age for notifications of the notification class. In this example, the expiration age is set to 6 hours. Undelivered notifications older than 6 hours will never be delivered.

Testing the Custom Delivery Protocol

Use the following steps to get the music store instance running with the custom delivery protocol:

1. | Build the MQDeliveryProtocol Visual Studio solution. This produces the custom delivery protocol assembly.
2. | Add the protocol and delivery channel declarations to your ICF. The ICF code in Listings 10.8 and 10.9 is available in the supplemental ICF, InstanceConfiguration-3.xml in the C:\SQL-NS\Chapters\10\SupplementaryFiles directory. Add this code to the main ICF (C:\SQL-NS\Samples\MusicStore\InstanceConfiguration.xml) either by typing it manually or by copying over the supplemental ICF.
3. | Add the declaration for the MQProtocol to the ADF. The ADF code in Listing 10.10 is available in the C:\SQL-NS\Chapters\10\SupplementaryFiles\ApplicationDefinition-15.xml supplemental ADF, which you can copy over the main ADF, C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml.
4. | Update your instance using the same steps you used before. (For reference, see steps 27 in the "Testing the FileSystemWatcherProvider in the Music Store Application" section, p. 251, in Chapter 8.) Make certain that you use the update_with_argument_key.cmd script, not the update.cmd script. Caution You must add both the new ICF code and the new ADF code (as described in steps 2 and 3) before updating the instance.
5. | After the update completes, start the SQL-NS service.

To test the new delivery protocol, you need subscriber devices configured to use the delivery channel that references it (MQChannel). When you completed the instructions in the "Adding Subscriber Devices for New Delivery Channels" section (p. 365) earlier, you added devices that use only the SMTP delivery channel. Now that you've added the MQChannel to the instance, you must run the AddMQSubscriberDevicesAndSubscriptions.sql script (located in C:\SQL-NS\Chapters\10\Scripts). This will add a set of subscriber devices that specify MQChannel as their delivery channel name (and a corresponding set of subscriptions that reference these subscriber devices). Complete the following instructions:

1. | Open AddMQSubscriberDevicesAndSubscriptions.sql in Management Studio.
2. | Take a look at the data in the records it adds. Notice that the new subscriber devices are of the TextMessageDevice type and specify MQChannel as the delivery channel name.
3. | Execute the script.

For the purposes of seeing notifications delivered by the custom delivery protocol, we'll use a simple client program that listens for messages on the message queue. When it receives a message, this program displays the message contents on the console. We'll start this program, submit some events into the music store application that match the subscriptions referencing the new subscriber devices, and then wait to see the listener program receive and display the resulting notifications.

Before you can test the custom delivery protocol, you need to build the client listener program using the following instructions:

1. | Navigate to the C:\SQL-NS\Samples\MusicStore\CustomComponents\Test\MQListener directory.
2. | Open the solution file MQListener.sln in Visual Studio.
3. | Build the solution. This creates the executable for the listener program, MQListener.exe, in the bin\Debug subdirectory.

Use the following instructions to exercise the custom delivery protocol and see the results:

1.
Make sure that the SQL-NS Windows service for the music store instance is running and then start the client listener program by typing the following command:

C:\SQL-NS\Samples\MusicStore\CustomComponents\Test\MQListener\bin\Debug\MQListener.exe

2.
Run the AddSongs program and connect to your music store database.

3.
Enter the following song data:
Album Title: Miles Davis and Horns
Artist Name: Miles Davis
Genre: Jazz
Song 1: Willie the Wailer
Song 2: Morpheus

4.
Check the Submit Events for Songs Added box and click the Add to Database button.

5.
About 30 seconds after you submit the events, check the command prompt window running the listener program. When the notifications arrive on the message queue, the listener program will print out their contents.

The messages printed by the notification listener show the header values created by the delivery protocol, as well as the notification bodies. Because the subscriber devices associated with the custom delivery protocol's delivery channel are of the TextMessageDevice type, the content formatter produces smaller notification bodies than the ones in the email notifications.

Note

To quit this MQListener program, press Ctrl+C in the command prompt window in which it is running.

Implementing Custom Delivery Protocols That Use HTTP

Many modern network protocols are based on the Hypertext Transfer Protocol (HTTP). These protocols all use HTTP as an underlying message transport, but implement different message formats and semantics.

Because network protocols based on HTTP are so prevalent, SQL-NS provides extra support for them, beyond the basic IDeliveryProtocol extension mechanism. Using the SQL-NS HTTP extension framework, you can implement custom delivery protocols based on HTTP, without having to reimplement the basic HTTP message exchange. Instead, you can use an HTTP implementation that SQL-NS provides and customize it for the particular HTTP-based protocol you want to support. To be clear, SQL-NS does not provide a built-in HTTP delivery protocol, but rather a partial implementation of a custom delivery protocol class that you can customize by writing a small amount of code (much less code than if you were reimplementing the fundamental HTTP message exchange).

To use this mechanism, you declare a custom delivery protocol (in a <Protocol> element of the ICF) that specifies HttpExtension as the class name. HttpExtension is the name of a SQL-NS built-in class that implements IDeliveryProtocol but calls out to custom code that you provide to build the HTTP message payload and parse responses from the delivery endpoint. You use HttpExtension in place of a custom delivery protocol class that you write on your own. You do not need to specify an assembly name in the protocol declaration when specifying HttpExtension as the class name.

You then declare a delivery channel in the ICF that references the custom delivery protocol name you specified in the <Protocol> declaration. In the delivery channel arguments, you specify the class name and assembly name for a protocol provider class that implements the IHttpProtocolProvider interface. The protocol provider is a class that you write. The HttpExtension class calls the IHttpProtocolProvider methods on the provider class to form the HTTP payload for messages that it sends out and to process HTTP responses received. Implementing IHttpProtocolProvider is much simpler than implementing an HTTP-based protocol from scratch using IDeliveryProtocol.

The HTTP implementation provided by HttpExtension is sophisticated: It uses multiple threads for asynchronous delivery and has been carefully optimized for maximum performance. Writing an equivalent HTTP implementation on your own would present a significant development challenge.

For more information on how to use HttpExtension and IHttpProtocolProvider, see the SQL-NS Books Online.


Restoring Your System

After running the tests in this chapter, you may want to undo some of the changes made to your system. Specifically, you may want to restart the SMTP service and remove the SQL-NS service account from the Administrators group. (Instructions for doing this are provided in this section.)

Before starting the SMTP service, it's important to clean up the SMTP pickup directory. Old messages left in this directory will be processed as soon as the service starts up, so we must delete the test messages created by the SQL-NS application to prevent them from being sent out. Use the following instructions to clean up the pickup directory and restart the service:

1.
Navigate to the SMTP pickup directory, usually located at C:\Inetpub\mailroot\Pickup.

2.
Delete any email files (files with a .eml extension) created by the SQL-NS application. If other email files are in this directory (created by other applications), you should leave those alone.

3.
From a command prompt, type the following command to start the SMTP service (you need to do this only if the service was running before you stopped it earlier):

net start SMTPSvc

To remove the SQL-NS service account from the Administrators group, use the following instructions. After you run these instructions, any attempts to deliver notifications with the SMTP delivery protocol will fail.

1. | From the Start menu, right-click My Computer and choose Manage from the context menu that appears. The Computer Management window opens.
2. | In the tree in the left pane of the Computer Management window, select the Local Users and Groups node.
3. | Double-click the Groups folder in the right pane.
4. | Double-click the Administrators group to open its properties page.
5. | Select the SQL-NS service account name in the list of group members.
6. | Click the Remove button. The service account should no longer appear in the list of members.
7. | Click OK to close the Administrators group property page.
8. | Click OK to close the Administrators group property page and then close the Computer Management window.


Summary

Delivery protocols are the components that handle notification delivery in SQL-NS applications. SQL-NS provides two built-in delivery protocols, File and SMTP. SQL-NS also provides an extensibility mechanism by which you can build your own custom delivery protocols. You may also be able to obtain custom delivery protocols from a vendor.

Delivery protocols are hosted in the distributor and implement various delivery methods that can be used to send notifications. Delivery channels represent specific delivery endpoints to which delivery protocols send notifications. Subscriber device records provide the link between notifications and the delivery channels that should be used to deliver them.

Delivery channels are declared in the ICF and reference delivery protocol names. Custom delivery protocols must be declared in the ICF before they can be used in delivery channels. To use a delivery protocol with a notification class, you must declare support for the protocol in the notification class declaration.

Notifications are grouped into notification batches when they are produced by the generator. The distributor selects notification batches that are ready for distribution and divides them into smaller groups called distributor work items. Each distributor work item contains the notifications in a batch going to a single delivery channel. Distributor work items form independent units of work for the application's distributors.

During the processing of a work item, the distributor reads notifications from the notifications table and passes them to a content formatter. The output of the content formatter is passed to a delivery protocol, along with information about the intended recipients and the values of any protocol fields. The delivery protocol sends the notifications to a delivery endpoint and reports their delivery status back to the distributor.

You can specify how failed notifications should be retried, by defining retry schedules in the ADF. If a notification delivery attempt fails, the distributor will retry the processing of the work item to which it belongs, on the schedule specified. Retry attempts on the work item will continue until all deliveries succeed, the retry schedule is exhausted, or the undelivered notifications expire. Notification expiration ages can be set in the ADF. The distributor maintains a log of all delivery attempts on each notification and makes this log available through views in the application database.

In this chapter, we added support to the music store application for the SMTP delivery protocol. To do this, we created a delivery channel that uses the SMTP protocol and added configuration information for the protocol to the NewSong notification class. We tested the application with the SMTP protocol and observed the resulting email notifications.

We also built a custom delivery protocol that uses Message Queuing to deliver notifications. We looked at the IDeliveryProtocol interface by which the SQL-NS engine interacts with delivery protocols and then wrote the code for a class that implements this interface. We tested the custom delivery protocol by integrating it into the SQL-NS instance and using a test program to listen for notifications on the message queue.

The work done in this chapter completes the functionality of the music store application. In the remaining chapters of this book we will look at how to tune this application for optimal performance and deploy it in a production environment.


Chapter 11. Debugging Notification Generation

IN THIS CHAPTER

  • Debugging and the Development Process

  • Inside the Notification Generator

  • Using the SQL-NS Debugging Facilities

When you're new to SQL-NS, you face a steep learning curve. Not only do you have to learn a whole new set of concepts and terms, but you also have to get the code right when building your first application. In this book, I've tried to isolate you from coding problems by presenting complete working examples that let you focus on the key SQL-NS concepts, without worrying too much about the specifics of the code. As a result, you've probably become accustomed to SQL-NS applications that work like they're supposed to, with little or no debugging required.

Unfortunately, when you start building your own applications, things may not go so smoothly. Like most SQL-NS developers, you will probably have to deal with problems in your code that keep your application from working as you'd expect. This chapter is about debugging the core SQL-NS application logic that you specify in the ADF. The tools and techniques described in this chapter will help you isolate and fix problems in your code.

Note

This chapter focuses on debugging code. It does not cover deployment and configuration problems, nor is it a general troubleshooting guide. For troubleshooting information, see Chapter 15, "Troubleshooting SQL-NS Applications."


Debugging and the Development Process

As described in Chapter 5, "Designing and Prototyping an Application," when building a new SQL-NS application, you should construct the prototype ADF one piece at a time (see the "Building the ADF Incrementally" section, p. 127). In the typical development process, you will add the schemas for the events, subscriptions, and notifications first, and then add the logic that connects them. The logic in the ADF is specified in rules of three types:

  • Event chronicle rules that maintain the state information stored in event chronicle tables

  • Event-triggered match rules that produce notifications for event-triggered subscriptions

  • Scheduled match rules that produce notifications for scheduled subscriptions

Note

As you may recall from Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State," you can declare subscription chronicles in the ADF, but there is no dedicated ADF element for declaring subscription chronicle rules. Generally, subscription chronicles are updated as part of the match rule logic.

All three types of rules are SQL statements that may contain errors (syntactical or semantic) when you first write them. This chapter is about finding and fixing those errors.

Syntax Errors in Rules

Syntax errors are usually easy to deal with. In most cases, if a rule definition contains a syntax error, you receive an error message when you compile the application. Compile-time error messages are printed to the console by nscontrol create. If you're using Management Studio to compile your application, the error messages are displayed in the progress dialog box of the New Notification Services Instance command. These compile-time error messages usually provide specific information that enables you to spot your mistakes quickly. After you identify the errors, you can fix them and then recompile.

In the few cases where syntax errors are not caught at compile time (this can sometimes happen if your rule logic constructs dynamic SQL code), you will get a syntax error message at the time the rule runs. In SQL-NS, runtime error messages appear as entries in the Application event log, which you can access by opening the Event Viewer from Administrative Tools. All event log entries generated by SQL-NS specify NotificationServices as the event source, and those relating to the execution of rules have event IDs in the 40004999 range.

Note

For more information on using the Application event log to view SQL-NS error messages, see the "Using the Application Event Log" section (p. 484) in Chapter 14, "Administering a SQL-NS Instance."

If your rules generate runtime error messages, you will have to fix them in the ADF and then update the instance to recompile the application. After the update completes, you can test your application again to verify that the runtime problems have been fixed. When you do this, it's important to look at the Application event log and make sure that no new error messages appear.

Semantic Errors in Rules

Semantic errors in rules are usually more difficult to track down than syntax errors. When you have a semantic error, the rule syntax may be correct (so you might not see an error from the compiler or the engine), but the rule does not do what you expect it to. Semantic errors usually manifest themselves as too few or too many notifications being generated. Based on the notification data generated when testing the application against sample events and subscriptions, you might deduce that a semantic error exists. Finding and fixing the semantic error can sometimes be a challenge.

When tracking down a semantic error, it usually helps to run the application's rules manually, outside the SQL-NS engine. Doing this allows you to look at the exact inputs and outputs, and examine what the rules are really doing. Running rules manually also allows you to examine the intermediate state of the application between rule invocations.

Because the rules are just SQL statements, you might be tempted to cut and paste them from the ADF into a Management Studio query window to run them manually. If you try this, you'll find that the rule statements won't run because they rely on views of the event and subscription data that don't exist (or aren't populated with the correct data) until the SQL-NS engine sets them up at runtime. Even if the views did exist, any rule that inserts into a notifications view will not work if invoked directly, because the triggers on the notifications views rely on internal state set up by the generator before they are normally invoked.

To get around these problems, SQL-NS provides a set of stored procedures that you can use to debug your rule logic. These stored procedures mimic the runtime environment set up by the SQL-NS engine and allow you to invoke the rules manually. This chapter shows you how to use these stored procedures effectively to debug your applications. Being comfortable with these debugging stored procedures will reduce the time and effort required to get the initial prototype of your application running correctly.


Inside the Notification Generator

All rules in a SQL-NS application are invoked by the generator component of the SQL-NS engine. Before running the rules, the generator is responsible for setting up the runtime environment that the rules rely on, including the views of the event and subscription data, and the state required by the triggers on the notification views.

In preparation for learning about the debugging stored procedures used to run rules manually, it's important to understand the way the generator works internally. This section describes the generator's processing model and how it affects the way that rules execute.

The Quantum Clock

In the generator model, the passage of time is seen not as a continuum, but rather as a series of distinct, fixed-length intervals. In SQL-NS terminology, each interval is called a quantum. A quantum defines a discrete "slice" of time, as shown in Figure 11.1.

Figure 11.1. The generator divides time into distinct quantums.

If the length of a quantum is 1 minute, the generator divides any block of time into a series of 1-minute intervals. Consider a 3-minute period, starting at 12:00. The generator does not see this as a continuous stretch of time but instead as a sequence of three quantums: 12:00 to 12:01, 12:01 to 12:02, and 12:02 to 12:03.

Note

The correct plural form of the English word, quantum is really quanta. However, in the context of SQL-NS, the form quantums is commonly used instead. You'll see this usage in the names of several stored procedures, tables, and columns in SQL-NS databases, as well as in some documentation.

Although the usage is not always consistent, quantums seems to be the more accepted form. For this reason, I use quantums instead of quanta throughout this book.

The generator maintains two clocks: a real-time clock that keeps the actual time and a quantum clock that tracks the current quantum. When the generator first starts, the two clocks are synchronized, and the first quantum spans the interval from the current real clock time to one quantum interval in the past. For example, if the generator starts at 12:00 and the quantum interval is 1 minute, the first quantum is the time period from 11:59 to 12:00.

Quantum Processing

The generator performs its processing one quantum at a time. When processing a quantum, the generator runs the application's rules against only the event batches that were submitted during the time interval that the quantum spans. Continuing the previous example, during the processing of the first quantum, the generator would run the chronicle and match rules against only the event batches that arrived between 11:59 and 12:00. The data from all other event batches would not be visible in the event views used in the rules. Thus, the quantum serves as a scoping mechanism for the event data against which the rules run.

Note

The quantum being processed always describes a time period in the past. The generator never processes a quantum that defines a time period encompassing the present real clock time, or extending into the future.

In addition to scoping event data, the quantum also scopes the scheduled subscriptions against which the rules run. During the processing of a quantum, the only scheduled subscriptions visible to the rule logic are those for which next scheduled evaluation time falls between the start and end times of the quantum. The subscription views will contain only the data for these subscriptions.

Note

Although the end time of one quantum is the start time of the next quantum, quantums are treated as non-overlapping periods. An event batch that arrives exactly on a quantum boundary will be processed in the quantum starting at that boundary (not in the quantum ending at that boundary). For example, an event batch arriving at exactly 12:00 would be processed in the 12:00 to 12:01 quantum, not the 11:59 to 12:00 quantum. Similarly, scheduled subscriptions with schedules on quantum boundaries will be processed during the quantums starting at those boundaries.

When it finishes running the rules against the events and subscriptions in a given quantum, the generator sleeps until the start of the next quantum. This is the cause of the latency you may have seen when waiting for the generator to process the events you submitted to the music store application in previous chapters' examples. If the events arrived during a time when the generator was sleeping, they would not be processed until the start of the next quantum. The duration of the quantum interval determines how long the delay might be. The default quantum duration for SQL-NS applications is 1 minute, but we have used a shorter duration of 15 seconds in all the examples so far (the setting that changed the quantum duration in the music store application is shown in Listing 5.10, p. 156, in Chapter 5). Thus, you may have experienced wait times of up to 15 seconds before the generator began processing new events in the music store application.

The "Optimizing the Quantum Duration" section (p. 434) in Chapter 12, "Performance Tuning," explains more about altering quantum durations and the effect this has on application performance. In summary, specifying a shorter quantum duration makes the application more responsive (because each generator sleep is shortened) but increases the processing overhead. In your own applications, you will need to select an appropriate quantum duration that takes into account the frequency of event arrival and balances the responsiveness requirements against the overhead involved. Chapter 12 discusses these trade-offs in more detail.

Discrepancies Between the Quantum Clock and the Real-Time Clock

Ideally, the quantum clock and the real-time clock always remain synchronized, with the current quantum always running from one quantum interval in the past to the current real clock time, as shown in Figure 11.2.

Figure 11.2. The quantum clock and the real-time clock in sync.

[View full size image]

For the clocks to remain synchronized, the work that the generator does for each quantum should take no longer than the quantum interval. However, processing the events and subscriptions in a quantum can sometimes take longer than the quantum interval when the system is under heavy load. Suppose, for example, that because of the volume of data, processing the events and subscriptions in the 12:00 to 12:01 quantum actually took 4 minutes to complete. The quantum processing would have started at 12:01 (in real clock time) and ended at 12:05. At this point, the generator would then process the next quantum, 12:01 to 12:02. In this state, we say that the quantum clock has fallen behind. If the quantum clock had been up-to-date, then, at 12:05, the generator would instead have begun processing the 12:04 to 12:05 quantum.

Even though the real-time clock is ahead of the quantum clock, the generator still processes one quantum at a time. Regardless of the real clock time, during the 12:01 to 12:02 quantum, the generator makes the events and subscriptions only from that quantum available to the rules. If additional events arrived after 12:02 or additional scheduled subscriptions were due for evaluation after 12:02, those would be processed during later quantums.

Although it always processes one quantum at a time, the generator does not sleep between quantums when the quantum clock falls behind the real-time clock. After finishing the processing of one quantum, the generator immediately starts the next one (without sleeping until a quantum boundary) until the quantum clock catches up to the real-time clock.

When configuring your own applications, always choose a quantum duration longer than the time required to process the events and subscriptions in a typical quantum. This allows the quantum clock to eventually catch up to the real-time clock, even if a few quantums occasionally require longer than the quantum duration to process. Figure 11.3 shows the quantum clock falling behind because of a long-running quantum and then catching up by evaluating the next few quantums more quickly.

Figure 11.3. The quantum clock falling behind and then catching up.

[View full size image]

Apart from long-running quantums, another reason the quantum clock can fall behind the real-time clock is generator downtime. If the generator stops running, the real-time clock continues to advance, but the quantum clock stops. When the generator starts up again, the quantum clock resumes from the last quantum processed, even though the real-time clock may be significantly ahead. The generator continues to process quantums one at a time, until the quantum clock catches up with the real-time clock. This behavior is the same whether the generator was taken down intentionally or was terminated abnormally.

Note

You can control how far the quantum clock is allowed to fall behind the real-time clock through settings in the ADF. These settings specify when the generator should start skipping the processing of old quantums to allow the quantum clock to catch up to the realtime clock. The "Quantum Limits" section (p. 437) in Chapter 12 describes these settings in detail.

Activities Within a Quantum

The processing of a quantum consists of several steps. First, the generator selects the events and subscriptions against which the rules should execute and populates views of the data in these events and subscriptions. Next, it executes the rules by invoking their SQL statements, which operate against the event and subscription views. For those rules that insert into a notifications view, the generator sets up the state that the triggers on the views rely on. After all the rules have executed, the generator cleans up this state, as well as the views that it populated for the quantum, and then goes to sleep.

This section describes these stages of quantum processing in more detail. It's important to understand how quantum processing works in order to debug your rules.

Event and Subscription Selection

The first step in processing a quantum is selecting the events and subscriptions against which the rules will be invoked. To understand the way the generator selects events, look at an event batches table (for example, the NSSongAddedEventBatches table for the music store application's SongAdded event class).

Each row in the table represents one batch of events. The StartCollectionTime column contains the time at which the event provider began submitting the events in the batch, and the EndCollectionTime column contains the time at which the batch was finally committed. In a given quantum, the generator selects event batches where the EndCollectionTime falls within the quantum's boundaries. In other words, it selects batches where the EndCollectionTime is greater than or equal to the quantum start time and less than the quantum end time. The generator selects event batches that meet this criteria from all the event classes in the application.

Note

The terms quantum start time and quantum end time refer to the start and end of the time interval that the quantum represents. They do not refer to the times that the generator starts and ends the processing of the quantum. The quantum start and end times are expressed in terms of the quantum clock, whereas the processing start and end times are measured by the real-time clock.

The generator selects event-triggered subscriptions based on the event classes of the selected event batches. Recall from Chapter 5 that the declaration of an event rule in a subscription class specifies the name of the event class that it matches. For example, the event rule in the NewSongByArtist subscription class specifies that it matches subscriptions against SongAdded events. The generator determines which subscription classes have event rules associated with the event classes of the selected event batches and then selects all the event-triggered subscriptions of those subscription classes. The event rules executed in the quantum operate against these event-triggered subscriptions.

Finally, the generator selects scheduled subscriptions by evaluating their schedules and choosing the ones due to be evaluated between the quantum start and end times. Each scheduled subscription specifies a start time and recurrence pattern, and the generator uses an efficient algorithm to determine from these which subscriptions have next evaluation times that fall within the current quantum. Because scheduled rules are not associated with any particular event class, the generator considers scheduled subscriptions from all subscription classes, regardless of the event classes represented in the selected event batches.

View Materialization

After the generator selects the event batches and subscriptions for the quantum, it materializes views of the event and subscription data. For each event class for which there is at least one selected event batch, the generator populates an event view. Event views have the same names as the event classes they represent. Each view contains the event data from the selected batches of one event class.

Similarly, for each selected subscription class, the generator populates a subscription view with the selected subscription data. Subscription views have the same names as the subscription classes they represent. In the case of subscription classes with event rules, all subscriptions appear in the view. For subscription classes with scheduled rules, only those subscriptions scheduled to be evaluated during the current quantum are included.

Now that you understand how the views are defined, it's worth going back to the definition of a match rule and revisiting how the views are used. For example, the match rule for the NewSongByArtist subscription class is written against views of SongAdded events and NewSongByArtist subscriptions as follows:

INSERT INTO [SongAlerts].[NewSong]
SELECT ...
FROM   [SongAlerts].[SongAdded] events
JOIN   [Catalog].[SongDetails] songs
    ON events.SongId = songs.SongId
JOIN   [SongAlerts].[NewSongByArtist] subscriptions
    ON subscriptions.ArtistName = songs.ArtistName

The references to the views are shown in boldface font in the FROM clause: SongAdded is a view of events of the SongAdded event class, and NewSongByArtist is a view of the subscriptions of the NewSongByArtist subscription class. The generator is responsible for ensuring that these views contain the correct event and subscription data for the quantum before executing the rule. The rule itself just matches whatever data it sees in these views. This model allows you to write rules without having to think about the scoping of data. In the rule logic, you do not have to be concerned with distinguishing new events from old ones, the ordering of event batches, or evaluating subscription schedules.

The reliance on the event and subscription views is one of the reasons that it's difficult to run rules manually by cutting and pasting their text into a Management Studio query window. The views are populated by the generator only at runtime, so their contents are unpredictable at other times. Later in this chapter, you'll see how to set up these views using the SQL-NS debugging stored procedures so that you can execute rules manually.

Rule Firing

After establishing the event and subscription views, the generator executes the rules. This section describes two important aspects of rule execution: the generator ordering models and the handling of rule failures.

Ordering Models for Rule Execution

The generator offers two models for the ordering of the event batches and scheduled subscriptions processed by the rules: quantum ordering and strict event batch ordering. The choice of ordering model affects the way the rules are executed.

When using quantum ordering, the generator processes all event batches in the quantum together. After placing the event data for all the event batches in the views, it first executes the event chronicle rules, followed by the event-triggered match rules, and finally executes all scheduled rules. The generator does not make any guarantees about the order in which rules for different event classes will be executed. In fact, rules for different event classes may be executed in parallel.

When using strict event batch ordering, the generator strictly observes the order of events and subscriptions within a quantum. For this reason, this processing model is also sometimes referred to as subquantum ordering. Using this model, processing in a quantum proceeds according to the following sequence:

1. | The generator populates the views of the subscription classes that have event rules but leaves all other views empty.
2. | The generator processes all scheduled subscriptions scheduled to be evaluated before the arrival time of the first event batch in the quantum. It does this by placing these subscriptions in the scheduled subscription views and leaving the event views empty. It then executes the scheduled rules. After executing the scheduled rules, the generator empties the scheduled subscription views.
3. | The generator looks for the unprocessed event batch in the quantum with the earliest EndCollectionTime and places its event data in the event view. It then runs the event chronicle rules and event-triggered match rules (the event-triggered subscription views were established in step 1). After executing the rules, the generator removes the event data from the event views.
4. | The generator then looks for scheduled subscriptions in the quantum scheduled to fire before the EndCollectionTime of the next event batch in the quantum. If any such scheduled subscriptions exist, the generator places their data into the scheduled subscription views, runs the scheduled rules, and then empties the scheduled subscription views.
5. | The generator then repeats steps 3 and 4 until no more event batches or scheduled subscriptions remain unprocessed in the quantum.

This sequence of operations has the following implications:

  • Event batches and scheduled subscriptions are evaluated in strict time order.

  • Event batches are processed one at a time.

  • During the processing of an event batch, only the event data from that event batch is visible in the events view. The data from other event batches in the quantum with earlier or later EndCollectionTime values is not visible.

  • All scheduled subscriptions that occur between event batches are processed togetherthat is, individual scheduled subscriptions are not ordered with respect to one another.

Quantum ordering is the more efficient of the two models, but it does not offer stringent ordering guarantees. In contrast, as its name suggests, strict event batch ordering guarantees that all event batches and subscriptions in a quantum will be processed in order. But, making this guarantee reduces the generator performance. You will need to select the appropriate ordering model based on the required semantics for your application.

In most applications, the more relaxed ordering semantics of the quantum ordering model are sufficient. This is the default mode in which the generator operates. If the semantics of your application demand strict event batch ordering, you must select that model instead, but this will have an impact on the performance of your application.

To specify the generator ordering model, you use the optional <ProcessEventsInOrder> element in the ADF. The value of this element is a Boolean that indicates whether strict event batch ordering should be used. The default value is false, meaning that quantum ordering is used. <ProcessEventsInOrder> is a subelement of the <ApplicationExecutionSettings> element in the ADF, in which general executing settings for the application are specified. Listing 11.1 shows how the <ProcessEventsInOrder> element is used to specify strict event batch ordering for an application.

Listing 11.1. The <ProcessEventsInOrder> Element in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    ...
    <ProcessEventsInOrder>true</ProcessEventsInOrder>
  </ApplicationExecutionSettings>
  ...
</Application>

Handling Rule Failures

If a rule fails to execute, the generator logs an error in the Application event log (usually with the event ID, 4045). By looking at the event log using the Event Viewer, you can determine which rules failed. Usually, the event message provides some indication of the reason for the failure. In general, after a rule failure, the generator continues processing subsequent rules.

The exception to this is that if an event chronicle rule fails, the generator does not execute the event-triggered match rules for that event class. This maintains consistency within the application because, in many cases, the event-triggered match rules rely on state maintained by the chronicle rules. If the chronicle rule fails to update the state of the application, the event-triggered match rules may produce incorrect notifications.

Cleanup

After all rules have executed, the generator cleans up the state associated with the quantum. It removes the event and subscription views and clears out any temporary tables that it used.

If the generator shuts down abnormally during the processing of a quantum (due to a power failure, for example), the quantum cleanup will happen the next time the generator starts. In general, quantum state is always persisted in the database, so the generator can recover from abnormal termination gracefully.


Using the SQL-NS Debugging Facilities

This section describes the tools and techniques you can use to debug the rules in your SQL-NS applications. After reading the material in the previous section, you should have a good understanding of the environment in which the rules run. The information in this section explains how to mimic that runtime environment so that you can execute rules manually to debug them.

Overview of Debugging Tools and Techniques

To debug rules in SQL-NS applications, the following approach is generally used:

1. | Disable the notification generator so that it does not try to execute rules as you are debugging them.
2. | Submit a batch of events to the SQL-NS application.
3. | Manually set the quantum clock to a time period that encompasses the arrival time of the event batch. SQL-NS provides stored procedures for setting the quantum clock.
4. | Use a SQL-NS stored procedure to create a new debugging quantum and set up its runtime environment, including materializing the views of the event and subscription data.
5. | Examine the event and subscription data in the views to see the exact input the rules will run against. At this point, you may run some queries that implement part of the matching criteria in your rules to test their operation. For example, if you're trying to debug a match rule that does not produce the right notifications, you can run a series of queries that implement parts of the FROM and WHERE clauses of the rule's SQL statements to determine which parts are responsible for the unexpected results.
6. | Use a SQL-NS stored procedure to manually execute the actual rules one by one. After each rule executes, you can look at the results it produced, as well as the general state of the application.
7. | If you find a problem with a rule, you can fix it in the ADF and then update the instance to apply the changes to the database. You can then either run the rule manually again to verify that it works correctly, or you can start the instance normally and allow the rule to be executed by the generator.

The key to this debugging approach is step 3, in which you set the quantum clock to a specific time. SQL-NS provides two stored procedures for setting the quantum clock: NSSetQuantumClock and NSSetQuantumClockDate. The first one sets the quantum clock back to a previously processed quantum. The second stored procedure sets the quantum clock to a specific quantum start time, which may or may not have been the start time of any previously executed quantum. Whichever stored procedure you use, a new quantum is created for debugging purposes. If you use NSSetQuantumClock, the new quantum will have the same start and end time (according to the quantum clock) of a previously executed quantum. If you use NSSetQuantumClockDate, the new quantum will have the start time you specify and an end time of one quantum interval later.

Note

All the SQL-NS debugging stored procedures explicitly check that the generator is disabled before performing any debugging operations. If the generator is not disabled, these stored procedures return an error.

To set up the runtime environment for executing rules after setting the quantum clock, SQL-NS provides a stored procedure called NSPrepareRuleFiring. This populates the event and subscription views for the debugging quantum and sets up any temporary tables and states used internally when executing rules. After calling NSPrepareRuleFiring, you can examine the views and other application state manually.

When you are ready to execute rules, you can invoke NSExecuteRuleFiring. This invokes the first rule that needs to be executed. After running the rule, you can check its results and the general state of the system. Calling NSExecuteRuleFiring again executes the next rule in the application. You can continue calling NSExecuteRuleFiring until all the rules have been executed.

In addition to the stored procedures for running rules manually, SQL-NS also provides a set of stored procedures you can use to debug problems with schedules in scheduled subscriptions. Subscription schedules can sometimes be confusing because you specify a start time and a recurrence pattern when you create the subscription. With only this information, it can be difficult to deduce which subscriptions will be evaluated at a particular time of day. SQL-NS provides two stored procedures, NSScheduledSubscriptionDetails and NSScheduledSubscriptionList, that use the generator's schedule-scanning algorithm to evaluate subscription schedules and return the list of subscriptions due to be evaluated during a given time period. If you're not seeing the expected results from your scheduled rules, you can use these stored procedures to determine which scheduled subscriptions should have been evaluated.

In the following sections, we use the techniques and stored procedures described here to debug the music store application. Although there shouldn't be any problems with the rules in the application at this point, walking through these steps will show you how to go about examining and executing rules manually. You can apply these techniques in your own applications if you need to debug your rules.

Preparing to Debug: Disabling the Generator and Submitting Events

Before we begin debugging, make sure that you have the music store instance created and running on your system. Unless you explicitly cleaned it up, it should still be there from previous chapters' work. If you did clean up the instance, make sure that you re-create it according to the instructions in Chapter 10, "Delivery Protocols," (see the "Re-creating the SQL-NS Instance" section, p. 362). After re-creating the instance, you must add subscribers and subscriptions againyou'll find instructions for doing this in the "Adding Subscriber Devices for New Delivery Channels" section (p. 365) in Chapter 10.

Before using any of the stored procedures to debug the rules in an application, you must first disable the generator to prevent it from trying to process quantums in the normal way while you are doing manual rule execution. To do this from the command line, you invoke nscontrol disable, passing the -generator argument to indicate that you are disabling just the generator component, not the whole instance. The batch file, disable_generator.cmd, in the C:\SQL-NS\Samples\MusicStore\Scripts directory invokes nscontrol disable this way. Note that we cannot use the disable.cmd script we've been using in previous chapters because that script disables the whole instance, rendering all components inoperable. We still need to submit events during our debugging, which isn't possible if the whole instance is disabled.

Take a look inside disable_generator.cmd. The nscontrol disable command is issued as follows:

nscontrol disable -n %INSTANCE_NAME% -generator %SQL_USER_INFO%

As before, the -n argument specifies the instance name, and the %SQL_USER_INFO% environment variable provides a SQL username and password if you're using SQL Server Authentication. The -generator argument specifies that only the generator should be disabled. As written, the command disables the generator for all applications in the instance (we have just one). To disable the generator for just a single application, in the case that your instance has more than one application, you can use the -application argument (in addition to the -generator argument) to specify an application name.

Note

For more information on disabling individual components, including techniques for doing these operations from Management Studio, see the "Enabling and Disabling Components" section (p. 492) in Chapter 14.

Run disable_generator.cmd from the C:\SQL-NS\Samples\MusicStore\Scripts directory, in a Notification Services Command Prompt, to disable the generator for the music store instance. You should see output indicating that the generator component is either in the Disabled or Disable Pending state.

Now we need to submit a batch of events that we can use to test the rules manually. Start the AddSongs program that we created in Chapter 8, "Event Providers," and log in to your SQL Server when the connection dialog box appears. If you need to, refer to the section "The AddSongs Program" (p. 243) in Chapter 8 for instructions on how to build and run the AddSongs program. Because we want AddSongs to act as an event provider, you should check the Submit Events for Songs Added box whenever you use AddSongs in this chapter.

Type the following song data into the AddSongs user interface (again, it's essential that you specify the artist name correctly; otherwise, the events won't match the subscriptions):

Album Title: The Girl in the Other Room
Artist Name: Diana Krall
Genre: Jazz
Song 1: Almost Blue
Song 2: Temptation

Make sure that the Submit Events for Songs Added box is checked; then click the Add to Database button. Verify that the events were submitted correctly using the following instructions:

1.
Open Management Studio and log in to your SQL Server.

2.
Open a new query window and issue the following query:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[NSSongAddedEventBatches]

The resultset shows a row for each submitted event batch. The last row should be the event batch just submitted. You can verify this by checking the StartCollectionTime and EndCollectionTime values (note that these are expressed in UTC). You can also verify the event data by looking at the rows in the events table that have the event batch ID of the last event batch row.

Note the event batch ID of the event batch you just submitted, as well as the EndCollectionTime value. You'll need these values when setting the quantum clock in the next section.

Setting the Quantum Clock

To run rules against the event batch we just submitted, we need to set the quantum clock to a time period that encompasses its EndCollectionTime. The SQL code shown in Listing 11.2 accomplishes this.

Listing 11.2. Setting the Quantum Clock

USE MusicStore
GO
DECLARE @EndCollectionTime DATETIME
DECLARE @QuantumStartTIme DATETIME

--Obtain the EndCollectionTime of the event batch
SELECT @EndCollectionTime = EndCollectionTime
FROM [SongAlerts].[NSSongAddedEventBatches]
WHERE EventBatchId = /*... fill in your event batch ID here ...*/

--Calculate a quantum start time of 5 seconds before the
--EndCollectionTime of the event batch.
SELECT @QuantumStartTime = DATEADD(second, -5, @EndCollectionTime)

--Set the quantum clock
EXEC [SongAlerts].[NSSetQuantumClockDate] @QuantumStartTime

The code in Listing 11.2 declares two local variables: one to hold the end collection time of the event batch, the other to hold the start time of the debugging quantum that will be created. After declaring the variables, the code obtains the EndCollectionTime value of the event batch by selecting from the event batches table. Pay particular attention to the WHERE clause in this query (shown in boldface font in Listing 11.2). This is where the event batch ID is specified.

You will find this code in the file C:\SQL-NS\Chapters\11\Scripts\SetQuantumClock.sql. Open this file in Management Studio and edit the WHERE clause of the query to specify the ID of the event batch you just submitted. For example, if your event batch ID was 7, the WHERE clause would read:

WHERE EventBatchId = 7

The next line calculates a start time for the debugging quantum by subtracting 5 seconds from the EndCollectionTime of the event batch. Because the music store application's quantum duration is 15 seconds, this ensures that the quantum will include the event batch. Technically, you can set the quantum start time to be the exact EndCollectionTime because the generator includes event batches that arrived exactly on a quantum start boundary, but adding a little buffer certainly doesn't hurt. The last line of Listing 11.2 invokes the NSSetQuantumClockDate stored procedure to set the quantum clock to the calculated start time.

Run the code in Management Studio now. The commands should complete successfully. You won't see any result data, but the quantum clock will be set so that a new debugging quantum will be created when we invoke NSPrepareRuleFiring in the next section.

Note

Instead of setting the quantum clock to a specific date and time with NSSetQuantumClockDate, you can use the NSSetQuantumClock stored procedure to set the quantum clock back to a previously executed quantum. You can then rerun the rules that were run when that quantum was originally processed. When rerunning a quantum this way, the event and subscription views are populated with the same data they contained during the original processing of the quantum.

To use NSSetQuantumClock, you need to first obtain the quantum ID of the quantum you want to execute. You can get this by looking at the quantum records in the NSQuantum1 table in the application schema.

NSQuantum1 maintains a record for every quantum that the generator executes. The StartTime and EndTime columns in this table indicate the start and end times of the quantum, according to the quantum clock. The RtcStartTime and RtcEndTime columns indicate the real clock times at which the quantum processing started and ended. Note that all time values are expressed in UTC. Identify the quantum you want to execute by looking at the start and end times and then make a note of the QuantumId. Pass this QuantumId to NSSetQuantumClock.

Alternatively, to find the quantum ID of the quantum in which a particular event batch was previously processed, you can use the view NSEventBatchView also in the application schema. This view contains a row for every event batch submitted to the application. Columns in the view indicate the event batch ID, the event class name, the start and end collection times, and the quantum ID of the quantum in which the batch's events were processed. Event batches that have not yet been processed have NULL in the QuantumId column, but those that have been processed will show a valid quantum ID.

Preparing for Rule Firing

After the quantum clock has been set, you call the NSPrepareRuleFiring stored procedure to create a new quantum, set up the views and other generator state, and get ready to execute the first rule. NSPrepareRuleFiring is located in the application schema (like all the other debugging stored procedures) and does not take any parameters, so you can invoke it as follows:

EXEC [SongAlerts].[NSPrepareRuleFiring]

Run this command in a Management Studio window after setting the quantum clock (using the code shown in Listing 11.2). Two resultsets are returned. The first resultset describes the new quantum and the rule that is set up to run. In the case of the music store application, the first rule that runs after a batch of SongAdded events is submitted is the chronicle rule that updates the event history. You see the name of this rule in the RuleName column of the first resultset. The second resultset indicates the names of any chronicles that the rule might affect. In this case, the resultset shows the SongAddedChronicle. If the rule was not a chronicle rule, the second resultset would have been omitted.

After running NSPrepareRuleFiring, you can examine the event and subscription views. Run the following query to see the data in the events view:

SELECT * FROM [SongAlerts].[SongAdded]

The data returned contains the song IDs of the two songs just added to the music store (when you invoked the AddSongs program earlier). Notice that we're selecting directly from the SongAdded view that the rule references, not from the events table, NSSongAddedEvents. This would not have returned the correct data had we not set the quantum clock and set up the generator state with NSPrepareRuleFiring. As you can see, the events view contains only the event data against which the rules should run; the events table may well contain other event data from previously processed event batches, but that data is not made visible to the rules.

Having properly set up the quantum state, you can also see the subscriptions against which the rule will run by executing the following query against the subscription class view:

SELECT * FROM [SongAlerts].[NewSongByArtist]

Again, this query selects directly from the view that the rule references, not from the underlying table.

At this point, if you were having problems with your rule logic, you could run test queries against the views. Because the views are set up as they would be during normal rule execution, these test queries would operate against the same, real input data that the rules themselves use. Although you could execute the actual rules as specified in the ADF, you should not do this directly for any rule that inserts into a notification view. The triggers on notification views rely on additional state that must be set up before they are invoked.

To properly execute rules that insert into a notifications view, use the NSExecuteRuleFiring stored procedure, as described in the next section. (NSExecuteRuleFiring is also the recommended way to run rules that don't insert into a notification viewfor example, chronicle rules.) Alternatively, you can execute the queries that obtain the data the rules insert into the notification views, without actually performing the inserts. If a match rule uses an INSERT-SELECT statement (as the music store application's match rules do), this is as simple as removing the INSERT line and running the SELECT statement on its own. Doing this gives you an opportunity to see the data the rule would normally insert into the notification view.

Executing Rule Firings

The NSExecuteRuleFiring stored procedure is used to execute the next rule in the quantum that is ready to run. Immediately after running NSPrepareRuleFiring as instructed in the previous section, the first rule to run will be the chronicle update rule. NSExecuteRuleFiring does not take any arguments, so you can invoke it as follows:

EXEC [SongAlerts].[NSExecuteRuleFiring]

NSExecuteRuleFiring returns up to three resultsets. The first resultset describes the rule that ran. The second lists any chronicles that might have been affected. In the case of rules that do not affect chronicles, this resultset is omitted. The third resultset describes any notification batches generated as a result of executing the rule. For the first chronicle rule, no notification batches are generated.

Note

NSExecuteRuleFirings isn't always capable of reliably determining whether a rule affected any chronicles. Sometimes you will see a chronicle table name listed in the second resultset, even if that chronicle was not affected.

After executing one rule, you can invoke NSExecuteRuleFiring again to execute the next rule in the quantum. In the case of our music store application, this will be the eventtriggered match rule for the NewSongByArtist subscriptions. This rule will generate notifications (because the events match some of the subscriptions), so when you invoke NSExecuteRuleFiring again, the third resultset will show a row describing a new batch of notifications.

Between each rule firing, you can examine the state of the system. This allows you to see the effects of each individual rule and isolate any problems that might exist. You can continue running NSExecuteRuleFiring repeatedly until there are no more rules to fire for the quantum. When this state is reached, the first resultset returned from NSExecuteRuleFiring indicates a NULL rule name.

Using the Scheduled Subscription Debugging Stored Procedures

The NSScheduledSubscriptionList and NSScheduledSubscriptionDetails stored procedures allow you to obtain information about the scheduled subscriptions that will be evaluated during a particular time period. Both stored procedures are created by the SQL-NS compiler in the application schema. NSScheduledSubscriptionDetails returns the subscription data for all subscriptions in a subscription class that are scheduled to fire between a given start and end time. NSScheduledSubscriptionList returns a list of all the scheduled subscriptions for a particular subscriber, across all subscription classes, that are scheduled to fire during a given time period.

Adding Scheduled Subscriptions

To see the scheduled subscription debugging stored procedures in use, we need to add some scheduled subscriptions to the application. To do this, we'll use the same SubscriptionLoader tool we used to test scheduled subscriptions in Chapter 6. Use the following instructions to add scheduled subscriptions with SubscriptionLoader:

1. | Invoke SubscriptionLoader.exe from the C:\SQL-NS\Tools\bin directory on your development machine.
2. | In the Connect to SQL Server dialog box, enter the name of your SQL Server and select an authentication mode. If you select SQL Server Authentication, enter the login name and password of the development account you created in Chapter 2, "Getting Set Up." Click OK to connect.
3. | In the main SubscriptionLoader dialog box, select the MusicStore instance, the SongAlerts application, and the NewSongByGenre subscription class (be careful not to select the event-triggered subscription class, NewSongByArtist, which is also offered as a choice).
4. | In the next section of the SubscriptionLoader dialog box, the Subscriber ID dropdown list shows the three subscribers we added to the music store instance. Select any of these subscribers.
5. | Enter values for the subscription fields in the data grid. For the GenreName field, enter Jazz, Rock, or Hip Hop. For the DeviceName field, enter WorkEmail and for the Locale field, enter en-US.
6. | Specify a subscription schedule in the bottom section of the SubscriptionLoader dialog box. In the Schedule Start Time field, select a start time 2 or 3 minutes into the future. Select your local time zone in the Timezone field (it should be preset to your local time zone, but you can change it as necessary). Note that the initial value shown in the start time field is the time at which the program was startedthis value is not updated, so you may have to add several minutes to it if you've had the program open for a while.
7. | Click the Add Subscription button. You should see a confirmation dialog box telling you that the subscription was successfully added.
8. | Repeat steps 4 through 7 to add subscriptions for several subscribers. Use a variety of GenreName values, but always enter WorkEmail and en-US for the DeviceName and Locale fields, respectively.
Using NSScheduledSubscriptionDetails

Listing 11.3 shows the SQL code that invokes NSScheduledSubscriptionDetails to obtain information about the NewSongByGenre scheduled subscriptions due to fire in a 10-minute period around the current time.

Listing 11.3. Use of the NSScheduledSubscriptionDetails Stored Procedure

USE MusicStore
GO
DECLARE @StartTime DATETIME
DECLARE @EndTime   DATETIME

SELECT @StartTime = DATEADD(minute, -5, GETUTCDATE())
SELECT @EndTime = DATEADD(minute, 5, GETUTCDATE())

EXEC [SongAlerts].[NSScheduledSubscriptionDetails] N'NewSongByGenre', @StartTime, @EndTime

The code declares local variables to hold the start and end times of a period that extends from 5 minutes before the current time to 5 minutes after the current time. It then calls NSScheduledSubscriptionDetails, passing the name of the NewSongByGenre subscription class and the start and end times. Note that all times are expressed in UTC.

The code in Listing 11.3 is in the file C:\SQL-NS\Chapters\11\Scripts\ScheduledSubscriptionDetails.sql. Load this file into Management Studio and run it. If you run it within 5 minutes of the start times of the scheduled subscriptions you added, the resultset returned will contain the scheduled subscription data. If the schedule times for the subscriptions you entered fall outside the 10-minute window specified in the script, add a new set of scheduled subscriptions with start times closer to the current time. You can then run this code again to see the subscription data.

Using NSScheduledSubscriptionList

Listing 11.4 shows the NSScheduledSubscriptionList stored procedure in use.

Listing 11.4. Use of the NSScheduledSubscriptionList Stored Procedure

USE MusicStore
GO
DECLARE @StartTime DATETIME
DECLARE @EndTime   DATETIME
SELECT  @StartTime = DATEADD(minute, -5, GETUTCDATE())
SELECT  @EndTime = DATEADD(minute, 5, GETUTCDATE())

EXEC [SongAlerts].[NSScheduledSubscriptionList] N'Emily', @StartTime, @EndTime

Again, the code calculates a start and end time by subtracting and adding 5 minutes to the current time. It then calls NSScheduledSubscriptionList, passing a single subscriber ID and the start and end times. This returns all the scheduled subscriptions associated with that subscriber, from all subscription classes, scheduled to fire during the given time period.

You'll find the code in Listing 11.4 in the file C:\SQL-NS\Chapters\11\Scripts\ScheduledSubscriptionList.sql. Open this in Management Studio. Notice that the script passes the value "Emily" for the subscriber ID argument to NSScheduledSubscriptionList. If "Emily" was not one of the subscribers for whom you added scheduled subscriptions, you'll need to edit the script. Replace "Emily" with one of the subscriber IDs you selected in step 4 of the instructions in the "Adding Scheduled Subscriptions" section (p. 411).

Run the script within 5 minutes of the start times of the scheduled subscriptions you entered. In the resultset, you should see the data for the subscriptions associated with the given subscriber ID.

Reenabling the Generator

As explained earlier, the debugging stored procedures described in this chapter require the generator to be disabled before they can be used. The section "Preparing to Debug: Disabling the Generator and Submitting Events" (p. 405) described how to disable the generator using the nscontrol disable command with the -generator argument. While the generator is disabled, you can run the debugging stored procedures, but the application does not process new event batches on its own.

At the end of a debugging session, you may want to reenable the generator to let the application resume normal operation. To enable the generator, you can invoke the nscontrol enable command with the -generator argument. The -generator argument used with nscontrol enable specifies that the command should apply only to the generator component (just as it did when used in the nscontrol disable command earlier).

In the C:\SQL-NS\Samples\MusicStore\Scripts directory, you'll find the batch file, enable_generator.cmd, that invokes nscontrol enable with the -generator argument. Take a look at the contents of this file to see how nscontrol enable is called (notice that the command-line arguments are identical to those in the nscontrol disable call you saw earlier). Run enable_generator.cmd to reenable the generator component on your system. Alternatively, you can invoke the enable.cmd script to enable the whole instance.


Summary

When you first build a new SQL-NS application, you may well have errors in the application logic specified in the ADF. Syntax errors are usually easy to find and fix because they typically show up at compile time. Semantic errors, on the other hand, are usually more problematic because, in most cases, they manifest themselves only through unexpected results at runtime. To track down semantic errors, it usually helps to run the application's rules manually, examine the state of the application at several intermediate stages, and run test queries to isolate problems.

Unfortunately, you cannot take rule text out of the ADF and run it directly against the database. Rules rely on runtime state set up by the generator at the time the rule is run; without this state in place, the rule execution will fail or produce unpredictable results. To run rules manually, SQL-NS provides a set of debugging stored procedures you can use to mimic the runtime state set up by the generator. Using these stored procedures, you can execute rules outside the generator, examine their inputs and outputs, and look at the intermediate state of the application between individual rule executions.

Before exploring the details of the debugging stored procedures, we looked at internals of the notification generator to understand the runtime environment in which it executes rules. The generator works off a quantum clock that divides continuous time into discrete, fixed-length quantums. Quantums scope the data that the generator sees at any given time. When processing a particular quantum, event rules execute only against event batches submitted during that quantum, and scheduled rules execute only against those subscriptions due to be evaluated during that quantum. In the processing of a quantum, the generator performs several operations: It selects events and subscriptions against which the rules will run, sets up views of the event and subscription data, executes rules, and then cleans up.

After looking at the generator internals, we used the debugging stored procedures to manually execute rules in the music store application. We disabled the generator, submitted an event batch, and then used the NSSetQuantumClock stored procedure to set the quantum clock to the time period in which the event batch was submitted. We then used the NSPrepareRuleFiring stored procedure to set up the runtime state for rule execution and used the NSExecuteRuleFiring stored procedure to invoke each rule. Finally, we looked at the NSScheduledSubscriptionDetails and NSScheduledSubscriptionList stored procedures that evaluate subscription schedules and determine which scheduled subscriptions will be evaluated during a given time period.

There weren't any problems with the music store application that needed to be debugged, but the debugging techniques we applied in this chapter can easily be used in other applications. These techniques will help you track down problems with your applications and get your prototype running quickly.


Part III: Optimization, Deployment, and Administration

IN THIS PART

Chapter 12 Performance Tuning

Chapter 13 Deploying a SQL-NS Instance

Chapter 14 Administering a SQL-NS Instance

Chapter 15 Troubleshooting SQL-NS Applications


Chapter 12. Performance Tuning

IN THIS CHAPTER

  • Overview of Performance Tuning in SQL-NS Applications

  • Streamlining Data Flow

  • Optimizing Notification Generation

  • Optimizing Notification Distribution

Up to this point, this book has been about building the basic functionality of a notification application using the SQL-NS platform. You saw the music store sample application go from initial design, through prototyping, to functional completeness.

This chapter is about taking a functionally complete application and making it perform as well as it can. The goal of performance optimization is to minimize the resources consumed by the application and maximize the volume of events, subscriptions, and notifications it can handle.


Overview of Performance Tuning in SQL-NS Applications

The work of performance tuning can be broken down into three categories:

  • Changes to the application's data flow

  • Optimization of the application's logic and tuning of the SQL-NS engine's execution settings

  • Configuration of specialized deployment hardware for the application

This chapter covers the first two of these categories. Chapter 13, "Deploying a SQL-NS Instance," covers configuring deployment hardware for optimal performance.

The first two categories encompass changes every application should undergo to realize the best possible performance. The last category includes some changes that apply only in specialized deployment scenarios. To draw an analogy to the automotive world, doing the work in the first two categories is like doing the basic tune-up on a car: You adjust the engine's air and fuel intakes, set the timings, and so on. The work in the last category is like outfitting the car for racing: You may install performance tires, a specialized exhaust system, and a turbocharger. Every car should be tuned-up to make it as efficient as possible for everyday use, but not every car needs to be outfitted for racing.

Because SQL-NS is such a flexible platform, different applications built on it may have significantly different performance bottlenecks. For example, in some applications, most of the work is in notification generation, whereas in others, the expensive processing may be in content formatting and notification delivery. Furthermore, some applications may have a high volume of events but relatively few subscriptions, and other applications may have the opposite. Because of these fundamental differences, it's difficult to provide general performance tuning guidance that applies to all SQL-NS applications.

The purpose of this chapter is to introduce to you the various SQL-NS controls that affect performance and to describe general performance tuning principles. The particular characteristics of your application will determine how you should apply the strategies described in this chapter. In some cases, instead of making specific recommendations, this chapter just describes various factors for you to consider when deciding on a performance tuning approach for your applications. Although performance tuning should always be done methodically, results aren't always predictable, so you should expect that some trial and error will be involved.

Note

The music store application developed in this book provides the context for some of the material in this chapter, but be aware that not every performance tuning setting or strategy covered applies to this application.

It's important to note that the benefits of the performance tuning changes described in this chapter may not be directly visible when testing an application with small loads. However, they will make an appreciable difference during large-scale testing and when the application is deployed in production.


Streamlining Data Flow

Consider the flow of data in a SQL-NS application:

  1. Subscription data is entered by users through the SMI.

  2. Event data flows into the application from the event providers.

  3. The generator produces notification data by matching events and subscriptions.

  4. The distributor reads notification data, formats it, and then delivers it.

The performance of the application is directly related to the efficiency of the data flow through it. The most obvious way to improve data flow efficiency is simply to move less data through the application. This section describes two strategies, data normalization and event prefiltering, that you can use to reduce the volume of data moving through a SQL-NS application.

Normalizing Data

In database design, normalization refers to structuring tables so as to avoid storing redundant data. In a normalized database schema, each data element is stored only once, uniquely identified by a primary key, and related to other data elements through foreign key relationships. Normalizing data leads to more efficient storage and helps maintain data integrity.

In SQL-NS applications, the principles of normalization can be applied to reduce data copying and thereby make data flow more efficient. Consider the way events are modeled in the music store application. Conceptually, each event is a description of a new song added to the music store, but rather than carry all the song properties (title, artist name, album name, and genre) with each event, we chose to store just the song ID.

The song ID is a foreign key into the songs table in the music store catalog. Based on that relationship (and other relationships to the artists, albums, and genres tables), we are able to obtain all the song properties without having to copy them into the SQL-NS events table. This leads to data flow efficiency because moving a single integer identifier (the song ID) is much cheaper than moving several strings for each event.

Note

For a review of the music store application's event schema design, refer to the "Events" section (p. 135) in Chapter 5, "Designing and Prototyping an Application."

This same technique can be applied to subscriptions and notifications as well. Anytime several rows in a table contain the same data, there is an opportunity to normalize that data into a separate table (where it is stored just once) and refer to it by a foreign key.

Normalizing Subscription Data

As an example of normalizing subscription data, consider the ArtistName field in the NewSongByArtist subscription class. It's likely that many subscriptions will have the same artist name. Instead of storing the artist name in the subscription data, an alternative design could have stored an artist ID in the subscription data and kept a separate table of unique artist names (with the artist ID as the primary key). Figure 12.1 shows this normalization of the subscription data. The artists table shown could be the artists table in the music store catalog, or a separate artists table maintained as part of the SQL-NS application data.

Figure 12.1. Normalizing the subscription data.

[View full size image]

Normalizing the ArtistName field enables the application to store less data for each subscription. A similar technique could have been used to normalize the GenreName field in the NewSongByGenre subscription class. The efficiency gained from doing this normalization does come at a cost. First, the match rules would become more complex because they would each have to join an additional table to obtain all the required subscription data. Furthermore, maintenance of the subscription data would become more difficult: For each subscription added, the application would have to ensure that there was a corresponding entry in the separate artists (or genres) table. The logic to do this could be implemented in the logic layer of the SMI, but there is always the possibility that this could be circumvented. Maintaining data integrity when a subscription is removed can also be a challenge because the application can remove the corresponding row in the artists or genres tables only if no other subscriptions refer to it.

In the particular case of the music store application, it's questionable whether normalizing the subscriptions would make a difference because the amount of duplicate data is small (just one string field). When designing the music store sample application, I chose not to normalize the subscription data because the gains did not seem worth the trade-off in complexity. In other applications, where the amount of redundant subscription data is much larger, normalization may be more worthwhile.

Tip

An often-stated motto of performance tuning experts is, "When in doubt, measure!" In other words, the best way to understand the performance characteristics of a system is to perform actual performance measurements. Measurements can turn up surprises. Often, the most thoroughly considered assumptions about performance issues turn out to be flawed. Chapter 14, "Administering a SQL-NS Instance," describes the SQL-NS reporting stored procedures and performance counters that can be used to obtain real performance data from a running SQL-NS system.

Normalizing Notification Data

Normalization can also be applied to the notification data, in some cases. If the notification data is merely a copy of data in other tables, you can refactor the notification class schema so that the notifications refer to the data in those other tables, instead of duplicating it. The content formatter then has to bring all the data together by joining notification data with the data in the separate tables, using foreign key relationships. Because this step often involves some custom processing, you almost always need to write a custom content formatter if you normalize your notification data.

In the music store application, the NewSong notification class schema contained fields for all the song properties. This data is a copy of the information stored in the music store catalog, and significant redundancy is likely among the rows in the notifications table. As we did for events, we could have stored just the song ID in the notification data and used this to refer to the additional song properties, as shown in Figure 12.2.

Figure 12.2. Normalizing the notification data.

[View full size image]

Doing this would have reduced the amount of data the generator has to write (and the distributor has to read) for each notification. However, this improved data efficiency would have come at the cost of content formatter complexity and overhead. The content formatter would need to do more work to obtain the song properties for the notifications. In the simplest implementation, this would mean an additional database read for each notification. A better implementation could have the content formatter cache the song properties in memory for the songs that it processes. As long as the cache's memory consumption is constrained, this approach would probably lead to fairly efficient performance over a large set of notifications.

The biggest drawback of normalizing notification data is that it may prevent the distributor from forming meaningful digest groups. In the examples shown so far, digesting groups notifications into a batch that specifies the same subscriber, subscriber device, and locale. However, you can specify additional digesting criteria by means of the <DigestGrouping> element. In the declaration of any notification field or computed field, you can include the <DigestGrouping> element to indicate whether the field should be considered when forming digest groups. The distributor looks at any fields that have a <DigestGrouping> element set to TRue, along with the subscriber, subscriber device, and locale, when it groups notifications for digesting. Notifications can be digested together only if they have the same value for all these fields. If you set the <DigestGrouping> element on one or more notification fields to true, you cannot normalize those fields out into separate tables and still maintain the digesting behavior. The distributor's digesting algorithms work only over the notifications table; they cannot consider data in other tables. Note that this is not a problem if you digest only on the standard columns (subscriber, subscriber device, and locale) and do not set <DigestGrouping> to TRue for any other fields.

Note

For more information on the <DigestGrouping> element, see the SQL-NS Books Online.

I chose not to normalize the notification data in the music store application in this book for the sake of simplicity and ease of explanation. Normalizing the notification data would have required the use of a complex custom content formatter from the start. However, in applications of your own, you should certainly consider normalizing your notification data as a way to improve data flow efficiency.

Prefiltering Events

The more events the event providers submit into an application, the more expensive each invocation of the chronicle and match rules becomes. One way to improve the data flow efficiency of an application is to reduce the number of events that the event providers submit, by filtering out events that won't match any subscriptions.

It is ultimately the job of the match rules to determine which events match the subscriptions, but event providers can also do simple checks to prevent obviously nonmatching events from ever reaching the generator. To understand how this might work, let's revisit the stock price application that we looked at in Chapter 3, "The Simplest Notification Application: Stock Quotes." Each subscription contains a stock symbol and a target price. A match occurs if an event and a subscription have the same stock symbol, and the current stock price in the event is greater than or equal to the target price in the subscription.

Events containing stock symbols for which no subscriptions exist will never generate matches. The event provider in this application can implement an event filter based on the set of stock symbols that appear in the subscription data: If there are no subscriptions for a particular symbol, no events for that symbol ever need to be submitted. Stock events that do contain symbols referenced in one or more subscriptions are submitted in the normal way, and the generator runs the application's match rule against them to generate notifications. Real stock feeds typically contain tens of thousands of stock events; this kind of filtering technique could drastically reduce the number of those events that the application actually has to process.

In the extreme, the entire logic of the match rule could be run as filtering code in the event provider. However, this would defeat the purpose of the generator infrastructure, which provides ordering semantics over event batches, rule firing reliability, and scheduled subscriptions. It's not recommended that you implement full subscription matching in your event provider, but if there are simple, cheap checks that you can do to intelligently filter your events, you may realize significant performance gains. Figure 12.3 illustrates the use of prefiltering to reduce the number of events submitted to a SQL-NS application. In the music store application, we didn't do any event prefiltering because the matching logic is almost trivial (just a simple comparison on the artist name or genre name)there isn't really a simpler check we could have done in the event provider.

Figure 12.3. Prefiltering event data.

[View full size image]

To implement prefiltering, you usually need access to the subscription data. For each subscription class, SQL-NS provides views over the subscription table that you can query to obtain the full set of subscription data. These are the same views you used to insert test subscription data in previous chapters' examples. The subscription class views contain a column for each field in the subscription class schema. The view for a subscription class is called NS<SubscriptionClass>View (where <SubscriptionClass> is the subscription class name) and is located in the application schema. For example, the view for the NewSongByArtist subscription class is called NSNewSongByArtistView and is located in the application's SongAlerts schema. You should always access subscription data through the views rather than query the subscription tables directly.

Note

Do not confuse the NS<SubscriptionClass>View with the view of the subscription data that you can reference in match rules. The NS<SubscriptionClass>View always contains the full set of subscription data in the subscriptions table. In contrast, the view referenced in the match rules is populated at runtime, has the same name as the subscription class, and contains only the subscription data against which the current rule execution should operate. In the case of scheduled subscriptions, this is a subset of the full subscription data.

In applications that make use of event history in chronicles, you need to be careful about the way you do prefiltering. In some cases, the application state may need to be updated for a particular event, even if that event does not match any subscriptions. Prefiltering purely on subscriptions can lead to incorrect application state in these cases.

Problems of this nature typically occur when an application has scheduled subscriptions that operate against event history. New scheduled subscriptions may use old event data, so the subscription data that exists at the time an event arrives is not always an accurate predictor of whether the event will be needed later.


Optimizing Notification Generation

In this section, we look at ways to improve the performance of notification generation in SQL-NS applications. As you've seen, the generator's primary job is to run the rules you define in the ADF. It is largely the performance of those rules that determines the performance of notification generation. Improving generation performance involves carefully examining the rules to find ways to make them run more efficiently. Generation performance can also be affected by a number of generator execution settings that you can adjust to the optimal values for your application. In this section, we first examine techniques for optimizing rules and then look at the generator execution settings.

Note

This chapter assumes that you still have the music store instance that you created in Chapter 10, "Delivery Protocols" (and used in the debugging exercises in Chapter 11, "Debugging Notification Generation"). We will use this instance to make some performance tuning adjustments. If you already cleaned up the instance, follow the instructions in the "Re-creating the SQL-NS Instance" section (p. 362) in Chapter 10 to get the instance back up and running. Make sure that you add subscriptions and submit some events that generate notifications; we need this data to be in the application to complete some of the performance-related steps described in this chapter. The section "Adding Subscriber Devices for New Delivery Channels" (p. 365) in Chapter 10 provides instructions for adding the required subscriber and subscription data, and the section "Preparing to Debug: Disabling the Generator and Submitting Events" (p. 405) in Chapter 11 describes some sample event data you can use.

Indexes and Query Optimization

All the rules that you define in the ADF are just SQL queries that operate over data in the events and subscription tables. Improving rule performance is really about tuning these SQL queries to make them run as efficiently as possible.

There are usually two aspects to optimizing SQL queries:

  • Indexing the referenced tables efficiently

  • Refining the queries so as to best leverage the indexes

Because rules operate against views of the event and subscription tables, proper indexing of those tables is the basis for improving rule performance. When the SQL-NS compiler creates the event and subscription tables, it creates indexes on some of the built-in columns used by the SQL-NS engine. But rule performance can usually benefit from additional indexes on some of the event and subscription class fields. The SQL-NS compiler has no understanding of the meaning of these fields, so it does not index them automatically. It is left to you to create the appropriate indexes over these fields, based on your knowledge of their semantics. This section describes the techniques you can use to identify the appropriate indexes for your application and the ADF elements you can use to create them.

Note

SQL query optimization is such a broad topic that entire books have been written about it. There is no way that this small section can provide adequate general coverage of this vast topic. Instead it offers some broadly applicable guidelines and focuses on aspects of query optimization specific to SQL-NS. To learn more about query optimization in general, the "Query Tuning" section in the SQL Server Books Online is a good place to start.

Viewing the Query Execution Plans for Rules

When working with simple queries, just looking at the query text is usually enough to tell you which columns in the referenced tables need to be indexed. Typically, those columns involved in joins are good candidates. In a more complex query, it helps to look at the query execution plan, which describes the way in which SQL Server executes the various parts of the query. The query execution plan (sometimes called the query plan for short) usually tells you which parts of the query are most costly to execute. This information can guide you in creating the appropriate indexes and adjusting the query to use them efficiently.

With the help of the SQL-NS debugging stored procedures (described in Chapter 11), you can use the SQL Server tools to examine the query plan for the rules in a SQL-NS application. To do this, you set the quantum clock to a time period that encompasses a previously submitted event batch, prepare the first rule firing, and then turn on the feature of Management Studio that displays execution plans for queries. With this feature turned on, you then execute the rules manually and observe the graphical query plans that Management Studio displays. Use the following instructions to try this with the music store application:

1.
From a Notification Services Command Prompt on your development machine, navigate to the music store sample's scripts directory by typing the following command:

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run disable_generator.cmd to disable the generator (this is required to run the debugging stored procedures).

3.
Open Management Studio and log in to your SQL Server.

4.
Open a new query window and issue the following query:

USE MusicStore
GO
SELECT * FROM [SongAlerts].[NSSongAddedEventBatches]

The resultset shows a row for each submitted event batch. Choose one of these event batches and remember its event batch ID.

5.
Load the file C:\SQL-NS\Chapters\11\Scripts\SetQuantumClock.sql. (The code in this file is shown in Chapter 11see Listing 11.2, p. 407.)

6.
In this file, you will see a comment that reads /*... fill in your event batch ID here ...*/ at the end of the WHERE clause in the first SELECT statement. Replace this comment with the ID of the event batch you chose in step 4.

7.
Run the script.

8.
Execute the procedure that prepares the first rule firing by typing the following code into a query window and then executing it:

EXEC [SongAlerts].[NSPrepareRuleFiring]

9.
Go to Management Studio's Query menu and select Include Actual Execution Plan to toggle the display of queries' execution plans. When this feature is enabled, the icon next to the menu item appears selected.

10.
Execute a rule firing manually by typing the following code into a query window and then executing it:

EXEC [SongAlerts].[NSExecuteRuleFiring]

Caution

When invoking NSExecuteRuleFiring with the option to include query plans turned on, you may receive a warning message that reads:

The query has exceeded the maximum number of resultsets that can be displayed
in the Execution Plan pane. Only the first 25 resultsets are displayed in the
Execution Plan pane.

You can ignore this message because the rule statement is always among the first 25 queries executed within the NSExecuteRuleFiring stored procedure.

11.
In the results pane of the query window, you will see three tabs, one of which is labeled Execution Plan. Click this tab and your window should look something like Figure 12.4.

Figure 12.4. Management Studio showing the execution plan for a rule.

[View full size image]

12.
The results pane shows the execution plan for each statement executed when you invoked NSExecuteRuleFiring. Some of these statements come from the internals of the SQL-NS stored procedures, but among them you will find the rule that executed (as specified in the ADF). Below the rule statement you will find the query execution plan, which you can study to understand how the query is executed.

13.
You can repeat steps 1012 to view the execution plan for each subsequent rule in the application.

Note

After you finish working with the debugging stored procedures, you must reenable the generator for the application to work in the normal way again. You can use the enable_generator.cmd script, described in the "Reenabling the Generator" section (p. 413) in Chapter 11, to do this.

Tip

If you hold the mouse pointer over an element in the query plan display, Management Studio will show a ToolTip that provides additional detailed information about that element. For general information about reading graphical query plans, see the "Displaying Graphical Execution Plans (SQL Server Management Studio)" topic in the SQL Server Books Online.

As an example of how the query plan can be used to identify inefficiencies in a query, let's examine the match rule for the NewSongByArtist subscription class. This match rule joins the events with the SongDetails view on song ID and the NewSongByArtist subscriptions on artist name. Listing 12.1 shows the match rule text, and Figure 12.5 shows portions of the resulting query plan (obtained using the steps described in this section).

Listing 12.1. The Match Rule Used in the NewSongByArtist Subscription Class

INSERT INTO [SongAlerts].[NewSong]
SELECT subscriptions.SubscriberId,
       subscriptions.DeviceName,
       subscriptions.Locale,
       songs.SongTitle,
       songs.ArtistName,
       songs.AlbumTitle,
       songs.GenreName
FROM   [SongAlerts].[SongAdded] events
JOIN   [Catalog].[SongDetails] songs
    ON events.SongId = songs.SongId
JOIN   [SongAlerts].[NewSongByArtist] subscriptions
    ON subscriptions.ArtistName = songs.ArtistName

Figure 12.5. The execution plan for the NewSongByArtist match rule.

[View full size image]

The query plan shows some inefficiencies. For example, the lookup of the subscription data requires an index scan, and the join with the song data requires a hash match. Together, these operations constitute 40% of the total query cost.

Caution

The cost percentages shown in the query plans are usually just estimates. Although in most cases they are fairly accurate, be aware that the real costs could deviate from those shown in the query plan.

Note

To make the query plan fit on the printed page, only the parts relevant to the discussion in this section are shown in Figure 12.5. Other parts of the query plan have been omitted. Breaks in the lines in Figure 12.5 indicate the places where parts of the full query plan have been removed.

Also, because the content of Figure 12.5 was derived from Management Studio screen captures, some of the labels on plan elements have been truncated. I recommend that you execute the rule and look at the query plan in Management Studio yourself, so that you can see it in its entirety. When doing so, look at the detailed information in the ToolTips that appear when you hover the mouse pointer over the plan elements.

If you do this, you may notice that the lookup of the event data executes over a table called NSCurrentSongAddedEvents, not the events table you're familiar with, NSSongAddedEvents. NSCurrentSongAddedEvents is an internal table used by the generator to materialize the SongAdded event data for the event batches being processed in the current quantum. There is a separate NSCurrent<EventClass>Events table for every event class in the application.

SQL Server has to resort to a scan over the subscription data because the subscriptions table is not indexed on the column being used in the match rule (the ArtistName column). Had there been an index on the ArtistName column in the subscriptions table, the expensive scan could have been replaced with a cheaper lookup mechanism. In the next section, we look how index creation statements are specified in the ADF and observe how the query plan changes when the right indexes are in place.

Creating Indexes in the ADF

Through dedicated ADF elements, you can specify indexes for the tables the SQL-NS compiler creates from event class and subscription class declarations. Listing 12.2 shows these elements used to create indexes that may help make the query plan we examined in the previous section more efficient.

Listing 12.2. Specifying Indexes in the ADF

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <Schema>
        <Field>
          <FieldName>SongId</FieldName>
          <FieldType>INT</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
      <IndexSqlSchema>
        <SqlStatement>
          CREATE INDEX SongIdIndex
          ON [SongAlerts].[SongAdded](SongId)
        </SqlStatement>
      </IndexSqlSchema>
      ...
  </EventClasses>

  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>ArtistName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
        ...
      </Schema>
      <IndexSqlSchema>
        <SqlStatement>
          CREATE INDEX ArtistNameIndex
          ON [SongAlerts].[NewSongByArtist](ArtistName)
        </SqlStatement>
      </IndexSqlSchema>
      ...
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>GenreName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
        ...
      </Schema>
      <IndexSqlSchema>
        <SqlStatement>
          CREATE INDEX GenreNameIndex
          ON [SongAlerts].[NewSongByGenre](GenreName)
        </SqlStatement>
      </IndexSqlSchema>
      ...
    </SubscriptionClass>
  </SubscriptionClasses>

  <NotificationClasses>
    ...
  </NotificationClasses>
  ...
</Application>

Below the <Schema> element in an event class or subscription class declaration, you can declare an <IndexSqlSchema> element. The <IndexSqlSchema> element contains one or more <SqlStatement> elements that define indexes. After the SQL-NS compiler creates the event and subscription tables, it executes these SQL statements to create the indexes. SQL-NS already creates a primary key clustered index on these tables (over the columns it adds to the schema), so you cannot create another clustered index in your <IndexSqlSchema> declaration (SQL Server allows only one clustered index per table). Beyond that, there are no restrictions: The <SqlStatement> elements can contain any valid CREATE INDEX statements, so all index options are supported. In fact, the <SqlStatement> elements within an <IndexSqlSchema> declaration can contain any valid T-SQL statements, but they are intended to encapsulate CREATE INDEX statements.

In the example shown in Listing 12.2, the event class declaration specifies an index over the SongId column. Notice that in the CREATE INDEX statement, the table name is given as SongAdded, the name of the event class itself. This might surprise you because, as you've already seen, the events table is called NSSongAddedEvents. In this context, you can use the event class name because the tables and indexes are created as follows: SQL-NS first creates the events table with the name of the event class, SongAdded, then applies the SQL statements in the <IndexSqlSchema>, and finally renames the table to NSSongAddedEvents. This allows you to specify indexes without having to know the internal naming scheme that SQL-NS uses (which may well change in future versions).

The subscription classes in Listing 12.2 also contain index declarations. The <IndexSqlSchema> element in the NewSongByArtist subscription class specifies an index over the ArtistName column. As in the event class, the subscription class name is used as the table name in the CREATE INDEX statement (SQL-NS uses the same renaming scheme to make this work). In the NewSongByGenre subscription class, the <IndexSqlSchema> declares an index over the GenreName column. Although we didn't actually observe the query plan for the NewSongByGenre match rule, based on what was learned about the structurally similar NewSongByArtist match rule, we can deduce that an index will be needed on this column because it's involved in a join.

Tip

You can create indexes over a chronicle table by specifying a CREATE INDEX statement directly in the <SqlStatement> elements in the chronicle declaration's <SqlSchema>. There are no dedicated elements for creating chronicle indexes. If chronicle tables play a significant role in your match rules, creating the right indexes over them may be just as important as indexing the event and subscription tables.

Testing the Effects of the New Indexes

In this section, we update the music store application with the index creation code shown in Listing 12.2 and then observe the effect on the query plan generated for the NewSongByArtist match rule.

The code in Listing 12.2 is supplied in the file C:\SQL-NS\Chapters\12\SupplementaryFiles\ApplicationDefinition-16.xml. To test it, you need to add this code to the main ADF in the music store sample's source directory, C:\SQL-NS\Samples\MusicStore\SongAlerts\ApplicationDefinition.xml. You can do so either by typing it in manually or by copying the supplementary ADF over the original. After you've added the code, update the instance as you've done before (disable the instance, stop the service, run the update_with_argument_key.cmd script, reenable the instance, and restart the service). For detailed instructions on updating the music store instance, see steps 27 in the "Testing the FileSystemWatcherProvider in the Music Store Application" section, p. 251, in Chapter 8, "Event Providers."

Unfortunately, adding indexes to the event and subscription tables causes the data in those tables to be removed during the update process. The addition of indexes appears to the SQL-NS compiler as a schema change, so it rebuilds the event and subscription tables. The old event data is discarded, but the compiler preserves the old subscription data.

Before re-creating the subscription tables, it renames the existing tables to NS<SubscriptionClass>SubscriptionsOld (where <SubscriptionClass> is the name of the subscription class). After the update completes, you can copy data from the old subscription tables into the new ones to preserve subscriptions.

To do this on your system, open the script C:\SQL-NS\Chapters\12\Scripts\CopyOldSubscriptionData.sql in Management Studio and run it. This script copies all the data from the old subscription tables into the new ones and then drops the old subscription tables. Dropping the old tables after copying the data out of them is an important step. If a subsequent update to the application causes the SQL-NS compiler to rebuild the subscription tables again, it will try to rename the existing ones to NS<SubscriptionClass>SubscriptionsOld once more. If the old tables from the previous update still exist, there will be a naming conflict and the update will fail.

Caution

Unfortunately, in the case of scheduled subscriptions, copying the old subscription data into the new tables isn't as simple as one INSERT-SELECT statement. The subscription schedule data is normalized into several tables, all of which are re-created during the update process. As it does for the main subscriptions table, the SQL-NS compiler renames the other tables by appending the "Old" suffix. The data in all these tables must be copied into the corresponding new tables while preserving all foreign key relationships.

The CopyOldSubscriptionData.sql script described in this section includes the code that restores all the scheduled subscription data for our scheduled subscription class, NewSongByGenre. Take a look at the script to see how this is done. Comments in the code explain each operation.

Because the old event data was discarded by the SQL-NS compiler during the update, we need to submit new batches of events to test the effects of the new indexes. As you did before, run the AddSongs program again to submit some events. You can enter any song data you choose, as long as the artist name is Miles Davis or Diana Krall (these are the artists for which subscriptions exist). Be sure to check the Submit Events for Songs Added box before clicking the Add to Database button to submit your song data.

With the subscriptions and events in place, run the instructions for viewing query plans listed in the "Viewing the Query Execution Plans for Rules" section (p. 425) earlier in this chapter. Make sure that you run step 2 (disabling the generator) if the instance was enabled after performing the update. Also, pay particular attention to steps 4 and 6. Because you've submitted new events, the event batch ID you will use in the SetQuantumClock.sql script will likely be different from the one you used before.

The rule we're interested in is the match rule for the NewSongByArtist subscription class, which is the second rule executed (you will have to perform step 10 twice). Portions of the new query plan for this rule are shown in Figure 12.6.

Figure 12.6. The new, more efficient, execution plan for the NewSongByArtist match rule that takes advantage of the new indexes.

[View full size image]

Note

Like Figure 12.5, Figure 12.6 is a truncation of the full query plan. To make the plan fit on the printed page, the sections that aren't relevant to the discussion in this section have been omitted. Omissions are indicated by breaks in the lines that connect plan elements.

The index scan has been replaced by a nonclustered index seek and the hash join has been replaced with a nested loops join. These are more efficient strategies and lead to lower overall query cost. As this example shows, by creating the right indexes, we can enable the query processor to execute the match rules more efficiently, and this ultimately leads to better notification generation performance.

Optimizing the Quantum Duration

Recall from Chapter 11 that the generator divides time into discrete units called quantums and that the quantum clock drives the generator's operation. The duration of a quantum is the polling interval on which the generator looks for new event batches to process. By default, the quantum duration is 1 minute, but this is adjustable through the ADF. In this section, we look at the effect that the quantum duration can have on generator performance.

The most visible impact of the quantum duration is latency in the processing pipeline. Consider an event batch submitted while the generator is sleeping. The event batch will not be processed until the start of the next quantum. The quantum duration determines how long that delay will be. In the music store sample application, we used a quantum duration of 15 seconds, instead of the default, 1 minute, to reduce the latency in the processing of events. Reducing the quantum interval makes the application more responsive because new event batches will be processed with less delay. The minimum quantum duration allowed is 1 second.

As you might expect, the increased responsiveness comes at a cost. First, a shorter quantum duration means that the generator checks for event batches more often, and the overhead involved in doing this check can become significant if it is done too frequently. Checking for event batches involves querying the database and writing records to the generator's internal tables, all of which can put load on the SQL Server.

Perhaps more important than the overhead cost is the reduced opportunity for batching that results from a short quantum duration. If the quantum duration is long, several event batches can accumulate in a single quantum. The generator can process these event batches together (assuming the application does not use strict event batch ordering) and amortize fixed costs over large sets of data. As the quantum duration is reduced, the number of event batches in a single quantum is likely to be less. This has a negative performance impact because processing event batches individually (or in small groups) is typically less efficient than processing several event batches together.

For these reasons, although reducing the quantum duration can make an application more responsive, doing so may well reduce the overall efficiency. The particular quantum duration appropriate for a given application depends on several factors, including

  • The expected arrival rate of events

  • Users' expectations of responsiveness and timeliness

  • The granularity of scheduled subscriptions

In some applications, events arrive frequently. The stock price application we built in Chapter 3 is an example of such an application. In other applications, events arrive only a few times a day. Our music store application would most likely fall into this category in reality. If events don't arrive frequently, it usually doesn't make sense to have a short quantum duration.

However, event arrival rate must be considered together with the second listed factor: the users' expectations of responsiveness and timeliness. In some applications, even if events occur infrequently, users want to be notified about those events as soon as they happen because there is a high value associated with the timeliness of the information. An example of this is an application that sends out breaking news headlines. In a given day, there may be only a few breaking news events, but when they occur, notifications should be sent out immediately. In these applications, a short quantum duration is appropriate, even though events are submitted infrequently.

The third factor in the list, the granularity of scheduled subscriptions, is also related to users' expectations of timeliness. The quantum duration actually limits the scheduled subscription granularity that can be supported. As described in Chapter 11, during a quantum, the generator executes scheduled rules against subscriptions whose schedules fall within the quantum's boundaries. So, a given scheduled subscription will be evaluated only during a quantum that encompasses its next evaluation time (as determined by its schedule). If the quantum duration is long, the real-time at which this quantum begins executing may be significantly offset from the actual time specified in the subscription schedule.

For example, consider a scheduled subscription with a schedule time of 12:02 in an application with a quantum duration of 5 minutes. Suppose that one quantum runs from 11:55 to 12:00, so the next quantum encompassing the scheduled subscription would be 12:00 to 12:05. Recall that when the quantum clock is up-to-date (in other words, not falling behind), the current quantum always runs from the current real clock time to one quantum duration in the past. So, the 12:00 to 12:05 quantum would only begin executing at 12:05 in real-time. Thus, the 12:02 scheduled subscription would be evaluated some time after 12:05.

Even though the SQL-NS subscription management API allows you to specify a subscription schedule in 1-second granularity, the effective granularity is determined by the quantum duration, as the example shows. When choosing a quantum duration for an application, you need to consider how accurately users expect their subscription schedules to be honored.

Tip

In your SMI, you can restrict subscription schedules to units of time equal to the quantum duration. For example, if the quantum duration were 5 minutes, the SMI could allow only schedule times that fall on 5-minute boundaries to be entered.

The three factors and their relative importance vary with the semantics of each application. When choosing a quantum duration for an application, you will have to weigh these factors against the efficiency concerns, as described in this section.

As you've seen in the music store ADF, to set the quantum duration, you use the <QuantumDuration> element. <QuantumDuration> is one of several optional elements that can be declared in the <ApplicationExecutionSettings> element to control the runtime behavior of the SQL-NS engine. Some of the other settings that can be declared in <ApplicationExecutionSettings> are described in the later sections of this chapter and in Chapter 13.

Listing 12.3 shows the quantum duration specified in the ADF. This example sets a quantum duration of 15 seconds. Notice that the quantum duration value is specified in the XSD duration syntax (explained in the sidebar "The XSD duration Data Type," p. 157, in Chapter 5).

Listing 12.3. Specifying a Quantum Duration in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    <QuantumDuration>PT15S</QuantumDuration>
    ...
  </ApplicationExecutionSettings>
</Application>

Note

If the <QuantumDuration> element is not specified, a default value of 1 minute is used.

Quantum Limits

As described in the "Discrepancies Between the Quantum Clock and the Real-Time Clock" section (p. 397) of Chapter 11, the generator's quantum clock can fall behind the real-time clock. This can happen if the processing that the generator does within a quantum takes longer than the quantum duration, or if the generator restarts after a period of downtime.

When the quantum clock falls behind, the generator keeps processing quantums until it catches up to the real-time clock. Depending on the workload, this may take a long time. While the generator is catching up, new events are ignored as old events are processed (because the generator proceeds one quantum at a time, in chronological order). In applications whose semantics demand that every single event be processed, this cannot be avoided. However, there are many applications in which old events can be skipped so that the generator can get to new events more quickly.

An example of this is the news headline application, mentioned earlier in this chapter. Imagine that the generator goes down, and when it comes back up, the quantum clock is several hours behind, with several hundred old quantums' news events to process. Headlines are of little value when they're old, so this application would serve its users better by skipping over the old events and processing the newest events sooner.

SQL-NS allows you to define quantum limits for the generator that constrain how far the quantum clock is allowed to fall behind before the generator starts skipping quantums. Quantum limits are always expressed as a number of quantums. When properly used, quantum limits can improve the performance of an application by avoiding the unnecessary processing of old and possibly irrelevant events.

To understand how quantum limits are used, consider an application that has a quantum duration of 1 minute and imagine the generator goes down for 1 hour, starting at 12:00. When it comes back up at 1:00 (in real-time), its quantum clock will be on the 11:59 to 12:00 quantum (as it would have been at the moment the generator went down). Had the quantum clock been up-to-date, at 1:00 the generator would have processed the 12:59 to 1:00 quantum. Therefore, the quantum clock is 1 hour, or 60 quantums, behind.

By specifying a quantum limitfor example, 20 quantumsthe application's developer can specify that the generator should skip old quantums until it is no more than 20 quantums behind. With this setting in place, the quantum clock would jump to the 12:39 to 12:40 quantum, and the generator would start its processing from there. All event batches submitted during quantums that were skipped (all those from 11:59 through 12:39), would never be processed.

Quantum limits are specified in the <ApplicationExecutionSettings> element of the ADF. Two separate quantum limits can be specified:

  • A chronicle quantum limit that defines the maximum number of old quantums for which the generator executes chronicle rules

  • A subscription quantum limit that defines the maximum number of old quantums for which the generator executes subscription class match rules

SQL-NS provides two separate quantum limit controls because, in some applications, the semantics that define when it is acceptable to skip the processing of old event batches may be different for chronicle rules than for match rules. It may be necessary to run the chronicle rules to update application state, even if the match rules are skipped. Using the two settings, you can limit the old quantum processing for these rule types separately.

As an example of how the limits work, picture an application that specifies a chronicle quantum limit of 40 and a subscription quantum limit of 10. If the generator gets more than 40 quantums behind, it will completely skip quantums (without running any rules) until it reaches the chronicle quantum limit of 40 quantums behind. Then, for the next 30 quantums, it will execute only the chronicle rules and skip the match rules. When it has caught up to the subscription quantum limit, 10 quantums behind, it will start executing both chronicle and match rules until it is completely caught up.

Listing 12.4 shows the ADF elements used to specify the two quantum limits.

Listing 12.4. Specifying Quantum Limits in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    ...
    <ChronicleQuantumLimit>40</ChronicleQuantumLimit>
    <SubscriptionQuantumLimit>10</SubscriptionQuantumLimit>
    ...
  </ApplicationExecutionSettings>
</Application>

The chronicle quantum limit is defined in the <ChronicleQuantumLimit> element, and the subscription quantum limit is defined in the <SubscriptionQuantumLimit> element. Both elements specify an integer number of quantums. The default value for the <ChronicleQuantumLimit> element (which is used if the element is not specified) is 1,440 quantums. With the default quantum duration of 1 minute, this amounts to 24 hours, or 1 day. The default value for the <SubscriptionQuantumLimit> element is 30 quantums. A value of zero in either element is taken to mean no limit.

Caution

Because the default non-zero values apply without the <ChronicleQuantumLimit> and <SubscriptionQuantumLimit> elements being declared, it is sometimes easy to forget that quantum limits are in effect. Be aware that quantum limits will always be enforced, unless you explicitly declare these elements with zero as values.


Optimizing Notification Distribution

This section describes various settings you can use to improve the performance of the distributor. In many applications, notification distribution can be the bottleneck in the SQL-NS pipeline, so optimizing the distributor's performance is important.

Almost all the settings described in this section affect the performance of a single distributor. Even with these settings tuned optimally, a single distributor may not provide the required level of performance in applications that generate extremely large volumes of notifications or that use slow delivery protocols. In these cases, employing additional distributors on separate machines to share the workload can improve distribution performance. Chapter 13 describes configuring multiple distributors in SQL-NS applications. If you use more than one distributor, you can apply the settings described in this section to each one individually.

Notification Batch Size

Normally, the notifications resulting from a single invocation of a match rule form a single notification batch. The batch size (number of notifications it contains) depends on the number of matches produced by the match rule.

You can override this behavior using settings in the ADF to control the notification batch size. With these settings, you can force the generator to break the results of a single match rule invocation into several small batches instead of one large batch. This can help distributor performance because it provides an opportunity for increased parallelism.

Note

Setting the notification batch size is supported only on SQL-NS Enterprise Edition. In Standard Edition, the notification batch size cannot be explicitly set.

The "Notification Batches and Distributor Work Items" section (p. 355) in Chapter 10 describes how the distributor breaks a notification batch into distributor work items. The work item is the smallest independent unit of processing in the distributor. The distributor processes multiple work items in parallel, to the extent possible.

To understand the performance impact of reducing the batch size, picture a single notification batch of 1,000 notifications, spread evenly across two target delivery channels. This batch would be broken up into two distributor work items (one for each delivery channel), each with 500 notifications. The distributor can process the two work items in parallel, but within each work item, the 500 notifications are processed serially.

Now imagine that instead of producing a single notification batch of 1,000 notifications, the generator is constrained to a maximum batch size of 200 notifications. With this setting, what was previously one notification batch would now form five notification batches, each with 200 notifications. These batches would each be broken into two distributor work items of 100 notifications. Figure 12.7 shows how the breakup into work items is affected by batch size.

Figure 12.7. The effect of batch size on the number of distributor work items produced.

[View full size image]

With the batch size constraint in place, the same workload, 1,000 notifications, is broken into 10 independent units of work for the distributor, instead of just 2. The distributor can parallelize these units of work and thereby achieve better performance. Within each unit, there are only 100 notifications to be processed serially, rather than 500.

Note

The extent to which the distributor can parallelize its work depends on the number of threads it has at its disposal. The number of threads is limited by system resources and can also be controlled via ADF settings, as described in the "Thread Pool Size" section (p. 444), later in this chapter.

To specify a notification batch size in the ADF, you use the <NotificationBatchSize> element in the <NotificationClass> declaration, as shown in Listing 12.5.

Listing 12.5. Specifying the Notification Batch Size in the ADF

<Application>
  ...
  <NotificationClasses>
    <NotificationClass>
    ...
    <NotificationBatchSize>100</NotificationBatchSize>
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

This example specifies a notification batch size of 100. Because <NotificationBatchSize> is a subelement of the notification class declaration, you can specify a different value for each notification class in your application. If you specify a value of zero for the <NotificationBatchSize> element, or if you omit the element entirely, no batch size limitation is imposed. All the notifications generated from a single rule invocation are treated as a single batch.

Note

Because the setting of the notification batch size is not supported on SQL-NS Standard Edition, if you include the <NotificationBatchSize> element in your ADF when using Standard Edition, you must supply a value of zero. Any other value will cause a compile error on SQL-NS Standard Edition.

The way SQL-NS interprets the value of the <NotificationBatchSize> element depends on whether digest delivery is used. If digest delivery is not used, the <NotificationBatchSize> element specifies the maximum number of notifications that a single batch can contain (as you might expect).

If digest delivery is used, the value of the <NotificationBatchSize> element specifies the number of notification batches to create for each rule invocation. If you specify a value of 5 for <NotificationBatchSize> when digest delivery is used, the generator will create five empty notification batches before invoking a match rule and then assign notifications to those batches in a round-robin fashion. As it does the assignment, it keeps notifications going to the same subscriber together in the same notification batch. The number of notifications in each batch depends on the number of matches produced by the rule and the extent to which multiple notifications are generated for the same subscriber.

Keeping notifications going to the same subscriber together in the same notification batch facilitates digest grouping later on (because digest grouping works only within each batch). If the generator interpreted the <NotificationBatchSize> value as it does when digesting is not used, the notifications going to a single subscriber may be split across two or more batches, and digesting would not be possible.

Caution

The different treatment of the <NotificationBatchSize> value, based on whether digesting is used, is often a source of confusion. When digesting is used, <NotificationBatchSize> does not specify a size at all, but rather a number of notification batches.

Using Multicast Delivery

Although multicast delivery has already been covered in previous chapters, its effect on distributor performance is worth revisiting. Multicast delivery reduces the workload placed on the application's content formatter by reusing the same formatted notification body for multiple recipients of the same notification data. The actual cost of formatting depends on the particular content formatter implementation, but the fewer times the content formatter has to be invoked, the more efficient the whole distribution pipeline will be.

Note

Multicast delivery is supported only by the Enterprise Edition of SQL-NS.

If an application sends the same notification data to many recipients, using multicast can significantly improve distributor performance. For detailed information on multicast delivery, see the "Aggregation Options: Digest and Multicast" section (p. 147) in Chapter 5.

Delivery Protocol Performance

The performance of delivery protocols can have a major impact on the overall distributor performance of a SQL-NS application. Notification delivery often involves communication with an endpoint over a network, and such communication may involve latency if the network (or endpoint) is slow. If the delivery protocol blocks distributor threads while waiting for network communication to complete, it can hold up the processing of subsequent notifications unnecessarily.

If you are implementing a custom delivery protocol, you can mitigate the effect of network latency by implementing asynchronous delivery. In the asynchronous model, the delivery protocol does not block the distributor while delivering each individual notification. Instead, the delivery protocol receives the notification data, places it on a queue for delivery, and then returns control to the distributor. The distributor can then proceed with the processing of subsequent notifications. The delivery protocol uses a worker thread (or a pool of worker threads) to asynchronously deliver the queued notifications. These worker threads incur the delivery delays but allow other work to continue in parallel. For more information on how to implement asynchronous delivery, see the discussion of the IDeliveryProtocol methods in the section "The Custom Delivery Protocol Interface" (p. 373) in Chapter 10.

Distributor Options

This section looks at a number of distributor-specific ADF settings that can affect distribution performance. As with all other settings described in this chapter, the particular values appropriate for a given application depend largely on that application's unique characteristics. It's difficult to give general recommendations on how to use these settings, but this section lays out some guidelines for choosing appropriate values.

Distributor Quantum

When it is not actively processing notifications and no new notification batches are ready for distribution, the distributor enters an idle state. In this state, it periodically looks for new notification batches to process and, if it does not find any, goes to sleep. After a time interval passes, the distributor awakens and again checks for new notification batches. The interval on which the distributor wakes up from the idle state to check for new notification batches is called the distributor quantum.

Note

It's unfortunate that the term quantum is reused in this context because the distributor quantum is really just a polling interval. The distributor quantum does not have any of the semantics of the generator quantum. There is no distributor quantum clock, the quantum does not act as a scoping mechanism, and there is no notion of falling behind or quantum limits.

The default distributor quantum is 1 minute. As you've already seen, the duration of the distributor quantum can be set in the ADF. In the applications we've tested so far, we've used a distributor quantum of 15 seconds, instead of the default. The shorter the distributor quantum, the more responsive the distributor will be to new notification batches that arrive while it is idle. However, as with the generator quantum, a shorter distributor quantum increases overhead because the distributor checks for work more often. Checking for work involves running a set of database queries, which can put a load on the SQL Server and consume resources in the SQL-NS engine.

The particular distributor quantum value appropriate for an application depends on how quickly the distributor should transition out of the idle state when a new notification batch becomes ready to process. Note that the distributor quantum value has an effect only when the distributor is idle (in other words, when it has no work to do and is periodically checking for new notification batches). When the distributor is actively processing distributor work items, it looks for new work at the start and end of each work item's processing and so the distributor quantum is of no consequence. When choosing a distributor quantum value for an application, you have to consider how often you expect the distributor to be idle and what the responsiveness requirements of the application are when the distributor is in this state.

To set the distributor quantum duration, you use the <QuantumDuration> element in the <Distributor> declaration, as shown in Listing 12.6.

Listing 12.6. Setting the Distributor Quantum in the ADF

<Application>
  ...
  <Distributors>
    <Distributor>
      ...
      <QuantumDuration>PT15S</QuantumDuration>
      ...
    </Distributor>
  </Distributors>
  ...
</Application>

Note

Although the element names are the same, do not confuse the <QuantumDuration> element in the distributor declaration (used to specify the distributor quantum) with the <QuantumDuration> element in the <ApplicationExecutionSettings> element (used to specify the generator quantum). The ADF schema would have been more symmetrical if the generator quantum were specified in the <Generator> element, but this is just an unfortunate inconsistency.

The distributor's <QuantumDuration> element uses the XML duration data type syntax, as does the generator's quantum duration element. The example shown in Listing 12.6 sets a quantum duration of 15 seconds. Because the distributor quantum duration is specified in the <Distributor> element, if you declare multiple distributors, you can provide a different quantum duration value for each one.

Thread Pool Size

When a distributor finds a work item to process, it assigns a worker thread to do the processing. The distributor can process several work items in parallel by using a separate worker thread for each one. You can control the maximum number of threads the distributor uses by specifying a thread pool size in the ADF.

The thread pool size is specified in the <ThreadPoolSize> element within the <Distributor> declaration, as shown in Listing 12.7.

Listing 12.7. Setting the Distributor Thread Pool Size in the ADF

<Application>
  ...
  <Distributors>
    <Distributor>
      ...
      <ThreadPoolSize>2</ThreadPoolSize>
      ...
    </Distributor>
  </Distributors>
  ...
</Application>

The default value for the <ThreadPoolSize> element is 3. In SQL-NS Standard Edition, you can specify any value between 1 and 3. In Enterprise Edition, there is no restriction on the thread pool size you can specify. Note that the value specified is a maximum. The actual number of threads that the distributor uses depends on the available resources at runtime.

Increasing the distributor thread pool size can improve the degree to which the distributor processes work items in parallel. But this is an important consideration only if the distributor will often have multiple work items to process at the same time. If an application produces notification batches infrequently and has only one or two delivery channels, the distributor thread pool size probably won't have much of an impact on the application's performance because the distributor would rarely have several work items to process in parallel.

Note

The distributor thread pool size does not affect the processing of an individual work item. Each work item is always handled by exactly one thread. The thread pool size affects only the processing of several separate work items in parallel.

Although parallel processing can improve distribution performance, as you might expect, it comes with a trade-off. The threads that the distributor uses come from a common thread pool within the SQL-NS engine's process. Consuming additional threads from this pool for work item processing leaves fewer threads available to other components in the process. This can be an issue, for example, if an asynchronous delivery protocol uses threads from the thread pool to deliver notifications. The distributor and this delivery protocol would be in competition for the same set of threads.

Also, increasing parallelism in the distributor puts more load on the SQL Server. Many distributor operations involve writing data back to the database (to report notification status, for example). Doing more distributor work increases the frequency of these operations. In fact, the SQL Server may eventually become the bottleneck limiting distribution performance. The higher the capacity of your database system, the more opportunity you will have to scale your distributor processing. Chapter 13 describes strategies for configuring a database system for the best performance.

In several deployed SQL-NS applications, the thread pool size setting is in fact used to limit the resources the distributor consumes, rather than increase them. The appropriate setting for any application depends on the degree of parallelism needed in the distributor and the number of other components in the application that utilize the thread pool.

Distributor Logging Options

The section "The Notification Distribution Views" (p. 360) in Chapter 10 describes views in the application database that provide information about the delivery attempts the distributor makes. The views are defined over tables in which the distributor logs status information as it processes notifications. This log information can be useful in tracking and debugging delivery problems, but the cost of writing it to the database can reduce distributor performance.

By default, the distributor writes an entry to the log table for each notification it processes, before passing the notification data to the delivery protocol. When the delivery protocol reports the status of the notification, the distributor updates this log entry with the status information. You can use settings in the ADF to change this default logging behavior and control how much information is actually written to the log. Turning off certain logging options can dramatically improve distributor performance.

The distributor logging settings are specified in the <DistributorLogging> element within <ApplicationExecutionSettings>. <DistributorLogging> can have the following subelements (all of which take Boolean values):

  • <LogBeforeDeliveryAttempts> Specifies whether the distributor should write an entry to the log for each notification before invoking the delivery protocol

  • <LogNotificationText> Specifies whether the distributor should log the notification text that the delivery protocol returns when reporting notification status

  • <LogStatusInfo> Specifies whether the distributor should log the extended status information that the delivery protocol returns when reporting notification status

Of these settings, the one that has the most drastic impact on performance is the first one, <LogBeforeDeliveryAttempts>. If you set this element to false, the distributor will write a log entry for a notification only after the delivery protocol reports its status. It will not write a log entry before invoking the delivery protocol, as it does in the default case. This effectively cuts in half the number of write operations (and database round-trips) that the distributor performs. In some cases, the reduction in write operations and database round-trips may be even greater because the distributor may group post-delivery status updates for several notifications and do them all in one operation.

The effect of the other two settings depends on the delivery protocol implementation. When a delivery protocol reports notification status, it calls the notification status call-back and passes a NotificationStatus object for each notification. Among the fields in this object are two that are used for logging purposes: NotificationText and StatusInfo. (See the description of the NotificationStatus object in the section "The Custom Delivery Protocol Interface," p. 373, in Chapter 10, for more information on these fields.) The <LogNotificationText> and <LogStatusInfo> elements control whether the information passed back in these fields is written to the database. Depending on how much information the delivery protocol returns in these fields, this may have a significant impact on performance.

All three elements that control distributor logging are optional; if you don't specify them, they default to true. Listing 12.8 shows the ADF code used to set them all to false.

Listing 12.8. Setting the Distributor Logging Options in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    ...
    <DistributorLogging>
      <LogBeforeDeliveryAttempts>false</LogBeforeDeliveryAttempts>
      <LogStatusInfo>false</LogStatusInfo>
      <LogNotificationText>false</LogNotificationText>
    </DistributorLogging>
  </ApplicationExecutionSettings>
  ...
</Application>

Setting all the distributor logging options to false provides the best performance but obviously reduces the amount of logging information made available through the notification distribution views. Note that even with all these options set to false, the distributor does provide some logging information. You must determine how much extra logging information you need in your applications and set these options appropriately. Often, the distributor logging options are set to true during an application's development phase, but then turned off when the application is deployed in production.

Additional Sample Code for this Chapter

If you want to see the code shown in Listings 12.3 through 12.8 in the context of a complete ADF, take a look at the file, C:\SQL-NS\Chapters\12\SupplementaryFiles\ApplicationDefinition-17.xml. This supplementary file adds the code in these listings to the most current music store ADF. You can update the music store instance on your system with the code in this file and see the effects of the adjusted settings.

The only difference between the code in the source file and the code shown in the chapter text is that the <NotificationBatchSize> value differs from the example shown in Listing 12.5. The source file specifies the value as zero because that is the only legal value on SQL-NS Standard Edition. Listing 12.5 shows a value of 100, which is valid only on Enterprise Edition. I want the source files that accompany this book to compile for all readers, regardless of which edition they are using, so I had to use a value that would work in both cases.


Summary

This chapter looked at several ways to improve the performance of a SQL-NS application. Some of the strategies described in this chapter can be applied during the design and prototype phase, and others are typically implemented after the basic functionality of the application has been built.

SQL-NS applications are driven by the flow of data, so improving the data flow efficiency within an application can bring about significant performance improvements. In the music store application, we used normalization to reduce the amount of event data that flows in. Similar techniques can be used to normalize the subscription and notification data. These techniques do improve data flow, but often require trade-offs in the complexity of the match rules and content formatters.

Prefiltering events is another strategy that can be used to improve data flow efficiency. Event providers may be able to easily filter out parts of an event stream that won't match any subscriptions. Doing this prevents the generator from unnecessarily evaluating subscriptions against those events and thereby improves the application's notification generation performance. Prefiltering is often done by performing simple checks against the subscription data, which can be queried through views that SQL-NS provides. When prefiltering events, you must make sure that chronicles in the application are updated correctly. Chronicle rules may need to run against event data, even if that data does not match any subscriptions. In such cases, the event data cannot be prefiltered.

To improve notification generation performance, you should optimize the chronicle and match rules that the generator runs. These rules are just SQL queries, and the standard SQL query optimization techniques can be used. This chapter explains how to obtain the SQL query plans for a SQL-NS application's rules with Management Studio and use them to identify performance problems. This chapter also covers the ADF elements that can be used to create indexes on the event and subscription tables.

In addition to tuning the application's rules, you can also improve notification generation performance by choosing an optimal quantum duration and setting appropriate quantum limits. This chapter describes how these settings can affect performance, explains the trade-offs involved, and illustrates the ADF elements used to specify them.

A number of settings can affect notification distribution performance. You can adjust the notification batch size to allow for more parallelism in the distributor, tune the distributor quantum duration to provide an optimal polling interval, set the distributor's thread pool size, and refine the distributor's logging behavior. In this chapter, we looked at the effects of all these settings, as well as the ADF elements used to specify them. In addition, this chapter described how the use of multicast delivery and asynchronous delivery protocols can boost the performance of the distributor.

Every application has unique characteristics and performance requirements that drive performance-tuning decisions. It is difficult to provide sound recommendations that apply to all applications, but the information in this chapter provides the background you can use to make the appropriate choices for your applications. When in doubt, take performance measurements to understand the performance characteristics of your applications and how you might best improve them. Chapter 13 builds on the material in this chapter by looking at ways to improve an application's performance by designing an optimal deployment environment.


Chapter 13. Deploying a SQL-NS Instance

IN THIS CHAPTER

  • Deployment Configurations

  • Configuring a Database System for Deployment

  • Vacuuming Old Data

  • Deployment Security

  • The Deployment Process

This chapter describes the process of deploying a SQL-NS instance in a production environment. In this chapter, we look at various deployment configurations that SQL-NS supports and how to choose the most appropriate one for your applications. We then look at the ICF and ADF elements used to configure a SQL-NS instance and its applications for the chosen configuration.

Ensuring the security of the SQL-NS instance is an important part of the deployment process. This chapter provides guidelines and techniques for securing your deployment, and highlights the places in which deployment security policy differs from the development environment.


Deployment Configurations

The components of a SQL-NS instance (the database, the SQL-NS engine, the SMI, and standalone event providers) can be deployed in several ways, on one or more servers. The section "Deployment of SQL-NS Instances on Multiple Servers" (p. 102) in Chapter 4, "Working with SQL-NS Instances," provides a brief overview of various multiserver deployment configurations. This section provides more detail on the supported configurations and describes the advantages of each. The particular one that is appropriate for your SQL-NS instance depends on the hardware resources available to you and the performance and reliability requirements of your applications.

Note

The terms server and machine are used interchangeably in this chapter. Both terms refer to a computer in a deployment environment.

The intent of this section is to illustrate the topology of the various deployment configurations (the layout of the machines involved and the functions performed by each). This section does not cover the configuration settings or deployment steps required to implement these configurations. That information is provided in the subsequent sections of this chapter.

Note

This section, and the rest of this chapter, covers deployment of only the server-side components of a SQL-NS instance. If your SQL-NS instance has client-side components (such as programs that serve as SMIs or standalone event providers), they will need to be deployed like any other client program.

Single Machine Deployment

This simplest deployment of a SQL-NS instance is one in which all the components are hosted on a single machine. This may be the configuration in which you've been running your SQL-NS instance during development. Figure 13.1 illustrates a single machine deployment.

Figure 13.1. A single machine deployment of a SQL-NS instance.

The simplicity of a single machine deployment has definite advantages: There is only one machine to administer, communication between components does not generate network traffic, and it's easy to move the instance to another machine if the need arises. However, there are some drawbacks as well. First, a single machine deployment is limited in its capacity to scale because all components on the machine share a common set of resources (disk, memory, CPU, and so on). More importantly, a single machine deployment can present a security risk. If one component becomes compromised and allows an unauthorized user to gain control of the machine, that user will have access to all the other components as well.

If your applications have only modest scale requirements, and you are comfortable with the security limitations, a single machine deployment may be appropriate. To implement a single machine deployment, you will need to install SQL Server, SQL-NS, and any components required to run your SMI (such as Internet Information Server [IIS] in the case of a web-based SMI) on the deployment server. You then follow the same steps used in the development process to create, register, and configure the SQL-NS instance.

Separating the Database Server

A common deployment configuration, used in the vast majority of SQL-NS applications, is to separate the SQL Server from the rest of the components of the SQL-NS instance. In this configuration, SQL Server is hosted on one machine, and all the other components are hosted on another machine. The databases associated with the SQL-NS instance and applications reside on the SQL Server machine. Figure 13.2 shows this deployment configuration.

Figure 13.2. A deployment of a SQL-NS instance with the SQL Server on a separate machine.

[View full size image]

Caution

When you separate the SQL-NS engine from the SQL Server as shown in Figure 13.2, the original SQL Server license no longer covers the SQL-NS engine. You will need an additional SQL Server license for the machine hosting the SQL-NS engine.

This configuration has the advantage of providing dedicated hardware to the SQL Server. The SQL Server machine can be fitted with high-performance hardware, such as RAID disk systems, fast networking, and lots of memory, and be tuned for maximum performance. The isolation of the SQL Server from the rest of the system is also desirable from management and security standpoints.

Note

This configuration is commonly used when the SQL Server is shared between the SQL-NS instance and other, unrelated applications that require a database.

As mentioned earlier in Chapter 4, the separate database deployment configuration is well suited to the way most SQL-NS applications work. Typically, a running application does about half its work in the database (updating chronicles and matching events with subscriptions) and half its work in the SQL-NS engine (submitting events and distributing notifications). The split of these two halves of the application processing onto separate machines provides dedicated resources for each.

To implement the separate database deployment, you install SQL Server on one machine (or use an existing SQL Server installation). On another machine, you install SQL-NS and any other required components, the same as you would if you were doing a single machine deployment. When you create the SQL-NS instance, you specify the name of the remote SQL Server. The SQL-NS compiler then constructs the instance and application database objects on this server. On the machine hosting the other components of the instance, you need to register the instance and install the SQL-NS engine. When you register the instance, you again specify the name of the remote SQL Server. The register command records the remote SQL Server name in the local Registry so that the engine can locate the instance and application database objects at runtime.

Scaling the SQL Server: Providing More Generator Horsepower

Because the majority of the generator's work is done in the database (it spends most of its time running chronicle and match rule queries), you can improve generator performance by deploying your SQL Server on a high-scale system. Generally, two strategies are used to increase the capacity of a SQL Server: scale up and scale out. Scaling up involves increasing the power of the machine on which the SQL Server runs. This can be done by adding more processors and using high-performance memory and disk systems.

Note

The generator component in the SQL-NS engine is just a coordinator for database activities and does not require many resources. Therefore, generator performance is largely unaffected by increasing the power of the system on which the SQL-NS engine runs.

In contrast to scaling up, scaling out involves partitioning the database over several separate SQL Servers. The database workload is shared among these SQL Servers, with each one handling a distinct portion of the total data. SQL-NS does not provide any direct support for partitioning, and it is generally difficult to implement partitioning of the SQL-NS database objects on your own. For this reason, it is recommended that you follow a scale up, rather than scale out, strategy for adding more power to your SQL Server.

Note

You can implement one simple form of partitioning fairly easily: just create two (or more) identical SQL-NS instances on various servers. Each one can handle a discrete subset of the subscriber base, and together they share the total workload. In such a system, the SMI directs each new subscription to one of the duplicate instances, according to some criteria. The event providers fan out events to all the SQL-NS instances. This strategy does provide scale out, but multiplies the administration and maintenance overhead.

The "Scalability and Very Large Database (VLDB) Resources" page (http://www.microsoft.com/sql/techinfo/administration/2000/scalability.asp) on Microsoft's website provides detailed information on scaling SQL Server.

Separating SMIs and Standalone Event Providers

If your SQL-NS applications have web-based SMIs, you need a web server to host them. In the deployment configurations we've looked at so far, the SMI web server runs on the same machine as the rest of the SQL-NS components, but it can also be separated onto its own machine.

The same arguments that justify separating the database from the rest of the SQL-NS instance also apply to separating the web server. It makes management easier (especially when the web server hosts multiple web applications) and provides security advantages. Also, if the web server is on a dedicated machine, that machine's hardware can be tuned and optimized to meet the particular performance needs of the web server.

Web servers are exposed to the Internet and therefore are often deployed outside the firewall that protects the rest of the infrastructure. Using a separate machine for the web server allows you to place it outside the firewall without worrying about exposing the rest of the SQL-NS instance to unauthorized users.

Standalone event providers may also communicate with Internet event sources (via either a push or a pull model), so they may also need to be outside the firewall. You can usually host these on the same machine as the web server. Figure 13.3 shows a deployment in which the SMI and standalone event providers have been separated onto a dedicated machine.

Figure 13.3. Using a dedicated machine to host the SMI and standalone event providers.

[View full size image]

Note

The single machine hosting the SMI web server depicted in Figure 13.3 can be replaced by a cluster or a server farm to increase the capacity of the SMI.

To implement a SQL-NS deployment with the web server and standalone event providers on a separate machine, you follow the same basic procedure used for the separate database deployment. However, you install the SMI and standalone event providers on their own machine rather than the machine hosting the SQL-NS engine. On the SMI machine, you must also install the SQL-NS Client components and register the instance.

Caution

If only the SQL-NS Client componentsthat is, not the SQL-NS Engine componentsare installed on a machine, a separate SQL Server license is not required. In the deployment depicted in Figure 13.3, SQL Server licenses would be required only for machines 1 and 3. The section, "Installing SQL-NS on the Deployment Machines" (p. 476), later in this chapter, provides instructions on installing only the SQL-NS Client components.

Scaling Out the SQL-NS Engine

In all the deployment configurations looked at so far, the SQL-NS engine runs on just one machine. The hosted event providers, generators, and distributors of the applications in the instance run in the engine, on that one machine.

SQL-NS supports separating the components that run in the engine onto separate machines. This enables scale out of the functions of the SQL-NS pipeline. Each machine runs its own copy of the SQL-NS engine (usually, a copy of the SQL-NS Windows service), hosting a subset of all the components. For example, the engine on one machine may host the event providers and generator, and the engine on another machine may host the distributor. Although the components may run on different machines, they are still considered part of a single SQL-NS instance.

Note

Only SQL-NS Enterprise Edition supports running the components of the engine on different machines. If you're using SQL-NS Standard Edition, the hosted event providers, generators, and distributors in an instance must all run on a single machine.

Although SQL-NS allows you to configure the components across machines in any combination you choose, in practice, the most common configuration has the hosted event providers and generator on one machine and the distributor on a separate machine. In most applications, the event providers and the generator component in the engine (which, as mentioned earlier in this chapter, are a coordinator for the generator's database activity) require the least resources. However, the distributor can often be a performance bottleneck, so separating it onto a dedicated machine can offer performance advantages if that machine is tailored to the distributor's particular needs.

For even more scale on the distribution end, an application can have multiple distributors and run each one on a dedicated machine. These machines then share the distribution workload of the application. Figure 13.4 shows this configuration. As described in the "Using Multiple Distributors" sidebar (p. 308) in Chapter 9, "Content Formatters," using multiple distributors in a single application makes sense (and is allowed) only if they are hosted on separate machines.

Figure 13.4. Separating the components of the SQL-NS engine onto separate machines.

[View full size image]

Caution

Every machine hosting the SQL-NS engine must be covered by a valid SQL Server 2005 license. When planning a scale-out deployment of the SQL-NS engine, you must ensure that you have the correct licenses.

To implement a SQL-NS deployment with the components of the engine running on different machines, you need to change the <SystemName> values specified in the components' configurations in the ADF. The section "Specifying the Server Configuration in the ICF and ADFs" (p. 477) describes how to do this. On each machine hosting the engine, you need to install the SQL-NS Engine components (as described in the section "Installing SQL-NS on the Deployment Machines," p. 476, later in this chapter). You also need to register the instance on each of these machines.

High Availability

You can use failover clustering to ensure high availability in your SQL-NS deployment. Both the SQL Server and the SQL-NS engine can be hosted on failover clusters for maximum availability. For more information on configuring SQL Server for high availability, see the "High Availability Resources" page on Microsoft's SQL Server website at http://www.microsoft.com/sql/techinfo/administration/2000/availability.asp. For general Windows clustering information, see the "Clustering Services" web page at http://www.microsoft.com/windowsserver2003/technologies/clustering/default.mspx.

When using a failover cluster to host the SQL-NS engine, use the virtual server name of the cluster in any place that a machine name is expected in the ADF. Specifically, the <SystemName> elements in the component declarations should specify the cluster's virtual server name. For more information on using clustering with the SQL-NS service, see the "High-Availability Configurations" topic in the SQL-NS Books Online.


Configuring a Database System for Deployment

In a SQL-NS deployment, the configuration of the SQL Server, and the instance and application databases, can have a dramatic impact on performance and reliability. Making the right choices about the databases at deployment time can also make administration tasks, such as backing up and restoring SQL-NS data, much easier later on. This section provides some overall guidance on setting up a database system and describes the SQL-NS settings that control the physical characteristics of the instance and application databases.

The choices you have to make for the database system in a SQL-NS deployment fall into three categories:

  • Hardware configuration of the machine hosting SQL Server

  • SQL-NS settings that control how the instance and application database objects are mapped to physical files

  • Other database settings that affect runtime behavior

This section provides general recommendations in all three categories, but the specific choices appropriate for your SQL-NS instance depend on its particular characteristics and the constraints of your deployment environment. The choices you make will determine the database settings needed in the ICF and ADFs before you deploy your SQL-NS instance.

Disks and Physical Database Files

Every SQL Server database is mapped to a set of physical files located in the filesystem of the SQL Server machine. SQL Server manages these files for you, reading from and writing to them when it executes queries. Through configuration settings, you can control how many files are created for a particular database, where those files are located, and how their sizes are maintained.

By carefully managing the physical storage of data, you can leverage the hardware on your SQL Server machine to achieve optimal performance. This section covers some basic principles of database physical storage and provides some recommendations for the configuration of your SQL Server machine. The next section describes the ICF and ADF settings used to manage the physical storage of the SQL-NS instance and application database objects.

Managing File Sizes

When SQL Server creates a database file, it preallocates a certain amount of disk space for it. This preallocated space is consumed as records are written to the database. If the database grows large enough, it will eventually require additional storage space, beyond the initial capacity of the file. When this occurs, SQL Server expands the file by allocating additional disk space for it.

When creating a database, you can optionally specify an initial size for its physical files, a growth increment (the amount by which the file should be expanded when SQL Server needs more space), and a maximum file size. If you don't specify these values, SQL Server defaults are used.

The process of expanding a data file to provide more capacity is usually CPU and disk intensive. In some cases, expanding a file may involve copying significant parts of the data from one area of the disk to another. Therefore, expanding files too frequently can reduce the performance of a database. For optimal performance, your database files should be configured so that they won't have to be expanded often. To achieve this, you can specify an initial file size that matches the amount of data you expect to store in the database and provide a growth increment that allows the file to expand proportionately to expected increases in data volume. In the section "Defining the Physical Storage of SQL-NS Database Objects" (p. 459), later in this chapter, we look at how these attributes can be specified for the SQL-NS databases files.

Separation of Data and Log

In the physical storage of a SQL database, one file is used to store the transaction log, and other files are used to store the actual data. A database can have several data files, but always has exactly one log file.

Almost every SQL transaction requires access to both the data and the log files: SQL Server reads from or writes to the data files to perform the actual data operation, and writes to the log file to record the transaction. If both the data and the log files are on the same physical disk, the disk heads may have to move between different areas of the disk surface for every transaction. Physically moving a hard disk head can be slow (relative to the speed of data transfer), so unnecessary movement is likely to reduce the rate at which transactions can be committed.

Separating the data and log files onto different physical disks can significantly improve performance. With the files on separate disks, contention is reduced, and the files can be accessed concurrently without having to constantly move the disk heads across the physical medium. Later sections in this chapter describe the configuration settings used to place the data and log files for the SQL-NS instance and application databases onto separate physical disks.

Note

The same principle applies to parts of the data that are often accessed concurrently. SQL Server allows you to create several data files for a single database and spread these across separate physical disks. You can then assign tables and indexes to specific files (or specific groups of files, as described in the "Using Filegroups" section, p. 459, later in this chapter). By doing this, you can eliminate disk contention for the most commonly accessed parts of the data.

Separating and Sizing tempdb

Every SQL Server has a temporary database called tempdb. This database is used to store temporary tables, hold intermediate results of join queries, and materialize the results of cursors. The tempdb is frequently accessed when running SQL-NS applications because the internal SQL-NS stored procedures use several temporary tables, joins, and cursors.

For the same reasons outlined in the previous section, it's recommended that you store the tempdb data files on a dedicated physical disk, separate from other database files associated with the SQL-NS instance. Also, it's a good idea to increase the size of the tempdb data file from its default to prevent unnecessary and costly file expansion.

The tempdb configuration is usually treated as a general SQL Server issue, not necessarily specific to SQL-NS. Therefore SQL-NS does not offer any settings that affect tempdb sizing. For more information on how to properly configure tempdb, see the "Optimizing tempdb Performance" topic in the SQL Server Books Online.

Hardware Considerations for the Database Machine

To implement the storage recommendations given in this section, the database machine in your SQL-NS deployment needs at least three physical disks. The first disk will store the instance and application database data files, the second will store the tempdb data files, and the third will store the log files. The capacity of these disks needs to be sufficient to store the expected volumes of data and logging information in your applications.

You may need additional disks if you plan to separate various parts of your data as well. This is usually necessary only in the largest of SQL-NS applications. In the next section, we look at how to assign the various SQL-NS application database objects (such as the event tables, subscription tables, and notification tables) to specific physical files.

Carefully plan the disk layout of your database machine and decide on the locations of the physical files in advance. Before deploying your SQL-NS instance, you may need to specify these file locations in your ICF and ADFs, as described in the next section.

Defining the Physical Storage of SQL-NS Database Objects

When a SQL-NS instance is compiled, the instance and application database objects are created in one or more databases. As you've seen in previous examples, the ICF and ADFs can specify the database and schema in which the objects are created, by means of the <Database> element. In addition to the database and schema, you can also specify how the individual objects are to be stored in physical database files. By organizing the physical storage appropriately, you can eliminate or reduce performance bottlenecks due to disk contention or file growth operations. This section describes the ICF and ADF elements used to specify physical storage for SQL-NS instances and applications.

Using Filegroups

When a database is created, its physical storage is specified in terms of filegroups. A filegroup is literally a group of files that are managed and administered together. A filegroup can consist of a single file, or several files, potentially on different physical disks. Each filegroup has a name that can be referred to in subsequent operations.

Individual database objects, such as tables, stored procedures, and triggers, are created on specific filegroups. The physical contents of a database object created on a particular filegroup are spread across the files in that filegroup. One filegroup in a database can be designated as the default filegroup; when creating an object in that database, the default filegroup is used unless another filegroup is explicitly specified.

When defining a SQL-NS instance or application, you can specify the filegroups on which the SQL-NS compiler creates individual objects. For example, you can designate a particular filegroup to be used for an event class; the compiler will then create all the objects associated with that event class (including the events table, the event batches table, and associated indexes) on the specified filegroup.

The following section, "Creating Filegroups" (p. 459), explains how filegroups are created. The section "Specifying Filegroups for SQL-NS Database Objects" (p. 464) describes the ADF elements used to assign various parts of a SQL-NS application to specific filegroups.

Creating Filegroups

When a SQL-NS instance is first compiled, the resulting database objects can be placed in existing databases, or new databases can be created for them. The choice is controlled by the values of the <DatabaseName> elements in the ICF and ADFs.

If the database specified in the <DatabaseName> element in an ICF or ADF does not exist at compilation time, the SQL-NS compiler creates it. Within the <Database> element, several subelements can be used to define the database's filegroups (and other physical storage attributes). These are shown in Listing 13.1. When the SQL-NS compiler creates a database, it uses the values specified in these elements to set up its physical storage.

Listing 13.1. The <Database> Element in the ICF Used to Define Filegroups

<NotificationServicesInstance>
  ...
  <Database>
    <DatabaseName>InstanceDB</DatabaseName>
    <SchemaName>NSInstance</SchemaName>
    <NamedFileGroup>
      <FileGroupName>Primary</FileGroupName>
      <FileSpec>
        <LogicalName>Primary1</LogicalName>
        <FileName>C:\DataFiles\Primary1.mdf</FileName>
        <Size>2MB</Size>
        <MaxSize>20MB</MaxSize>
        <GrowthIncrement>20%</GrowthIncrement>
      </FileSpec>
      <FileSpec>
        <LogicalName>Primary2</LogicalName>
        <FileName>C:\DataFiles\Primary2.mdf</FileName>
        <Size>2MB</Size>
        <MaxSize>20MB</MaxSize>
        <GrowthIncrement>20%</GrowthIncrement>
      </FileSpec>
    </NamedFileGroup>
    <NamedFileGroup>
      <FileGroupName>SecondaryGroup</FileGroupName>
      <FileSpec>
        <LogicalName>Secondary1</LogicalName>
        <FileName>C:\DataFiles\Secondary1.mdf</FileName>
        <Size>2MB</Size>
        <MaxSize>20MB</MaxSize>
        <GrowthIncrement>20%</GrowthIncrement>
      </FileSpec>
      <FileSpec>
        <LogicalName>Secondary2</LogicalName>
        <FileName>C:\DataFiles\Secondary2.mdf</FileName>
        <Size>2MB</Size>
        <MaxSize>20MB</MaxSize>
        <GrowthIncrement>20%</GrowthIncrement>
      </FileSpec>
    </NamedFileGroup>
   <LogFile>
      <LogicalName>Log</LogicalName>
      <FileName>E:\LogFiles\Log.ldf</FileName>
      <Size>10MB</Size>
      <MaxSize>60MB</MaxSize>
      <GrowthIncrement>20%</GrowthIncrement>
    </LogFile>
    <DefaultFileGroup>SecondaryGroup</DefaultFileGroup>
    <CollationName>SQL_Latin1_General_CP1_CI_AS</CollationName>
  </Database>
  ...
</NotificationServicesInstance>

As used in Listing 13.1, the <Database> element defines a new database to be created by the SQL-NS compiler. It does not simply specify an existing database, as other examples (including the music store sample) have done. When used this way, the <Database> element may contain one or more <NamedFileGroup> elements that declare filegroups to use for the physical storage of the data. The example shown in Listing 13.1 declares two filegroups, one named Primary and another named SecondaryGroup. SQL Server mandates that every database must have one filegroup named Primary. Databases can optionally have other filegroups, and those can be given any name.

Each <NamedFileGroup> declaration consists of a name for the filegroup and a series of <FileSpec> elements that define the individual files in it. The example in Listing 13.1 uses two files for each filegroup. Each <FileSpec> element provides a logical name for the file, a physical filename (including a drive letter and path), and the file size attributes.

The <LogFile> element specifies the same attributes for the database's log file. Notice that the log file isn't part of any filegroup. Filegroups are used only for data files. The log always consists of a single file, declared separately.

The <DefaultFileGroup> element specifies which filegroup should be used by default when creating objects in the database. All SQL-NS instance objects (such as the subscribers and subscriber devices tables) are created on the default filegroup of the instance database. However, in the ADF, you can override the default and choose other filegroups for particular application database objects, as described in the following section.

Note

The <Database> element is also used to set the database collation, which determines the character set and sort order used for the data. The <CollationName> element specifies the name of the database collation to use. See the SQL-NS Books Online for more information on this element.

Notice that in Listing 13.1, the filenames for the data files and the log file specify different drive letters. Assuming these drive letters correspond to different physical disks, this ensures the separation of the data from the log. When configuring a SQL-NS instance for deployment, the specified filenames and paths must correspond to the disk layout of the deployment database server.

Tip

You can use parameters to specify the drive letters and paths in the ICF and ADFs. This allows you to change them without having to edit the code. For more information on parameters, see the "Parameters" section (p. 104) in Chapter 4.

This book's music store sample instance (and all its other samples) used the same database to store both the instance and the application database objects. In other words, the <Database> element in both the ICF and ADF specified the same value for <DatabaseName>. However, application database objects can also be stored separately from the instance database objects, in a different database, by specifying a different <DatabaseName> value in the ADF than the ICF. In fact, every application within an instance can store its database objects in a different database.

The same elements shown in Listing 13.1 can be used within a <Database> element in an ADF (the syntax is exactly the same as the ICF example) to specify how a new database should be created for the application's database objects. But, this makes sense only if the <DatabaseName> element in the ADF specifies a database name different from the associated ICF and the database does not already exist at compile time.

Caution

If you specify the same database for both the instance and the application database objects and the database does not exist at compile time, you should specify the physical storage attributes only in the ICF. In the ADF, specify the database and schema name as though the database already exists. In fact, when the application's database objects are created, the database will exist, because the compiler creates it while compiling the ICF first. Physical storage attributes specified in the ICF or ADF are ignored if the database already exists.

The elements shown in Listing 13.1 are relevant only if the database specified in the <DatabaseName> element does not exist at compilation time. If the database does exist, only the <DatabaseName> and <SchemaName> elements should be presentall other elements should be omitted. If the <NamedFileGroup>, <LogFile>, <DefaultFileGroup>, or <CollationName> elements are specified when the database already exists, they are ignored.

When a SQL-NS application's database objects are deployed into an existing database, you cannot create new filegroups via ADF elements, but you can still assign parts of the application to existing filegroups, as described in the following section. These filegroups must have been previously created in the database. See the sidebar "Creating Filegroups With T-SQL" (p. 463) for an example of the T-SQL syntax used to specify filegroups when creating databases outside of the SQL-NS compiler.

Creating Filegroups With T-SQL

Generally, the filegroups that make up the physical storage of a database are specified when the database is created. If you're creating databases outside of the SQL-NS compiler, you can define filegroups in a CREATE DATABASE statement. For example, the following CREATE DATABASE statement creates the music store database with two filegroups:

   CREATE DATABASE [MusicStore]
   ON PRIMARY
   ( NAME = Primary1,
       FILENAME = 'C:\DataFiles\Primary1.mdf',
       SIZE = 2MB,
       MAXSIZE = 20MB,
       FILEGROWTH = 20% ),
   ( NAME = Primary2,
       FILENAME = 'C:\DataFiles\Primary2.mdf',
       SIZE = 2MB,
       MAXSIZE = 20MB,
       FILEGROWTH = 20% ),
   FILEGROUP SecondaryGroup DEFAULT
   ( NAME = Secondary1,
       FILENAME = 'C:\DataFiles\Secondary1.mdf',
       SIZE = 2MB,
       MAXSIZE = 20MB,
       FILEGROWTH = 20% ),
   ( NAME = Secondary2,
       FILENAME = 'C:\DataFiles\Secondary2.mdf',
       SIZE = 2MB,
       MAXSIZE = 20MB,
       FILEGROWTH = 20% )
   LOG ON
   ( NAME = 'Log',
       FILENAME = 'E:\LogFiles\Log.ldf',
       SIZE = 10MB,
       MAXSIZE = 60MB,
       FILEGROWTH = 20% )

The music store database used in this book's sample is in fact created with a statement much like the one shown here. You'll find this statement in the file C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql on your system. The exact file paths used in the script differ from the ones shown here (the ones in the script are customized for the install location you chose), but the number, names, and definitions of the filegroups are the same.

Like the example in Listing 13.1, the statement defines two filegroups. The first is called PRIMARY (as required by SQL Server) and encompasses two files. The second filegroup in this example is called SecondaryGroup, and, like the PRIMARY filegroup, also encompasses two data files.

The SecondaryGroup filegroup is designated as the default filegroup for the database by means of the DEFAULT keyword after its name. All objects subsequently created in the database are created on the default filegroup unless another filegroup name is explicitly specified.

The filegroup definitions each consist of two file specifications. Each file specification provides a logical name, a physical filename, the initial size, the maximum size, and the growth increment. Sizes and growth increments can be specified in several ways. The SQL Server Books Online provides complete details in the reference documentation for the CREATE DATABASE statement.

The CREATE DATABASE statement also specifies attributes for the physical storage of the database's transaction log. Much like the data file specifications, the log file specification includes a logical name, the physical filename, the size, the maximum size, and the growth increment.

Specifying Filegroups for SQL-NS Database Objects

The <FileGroup> element can be used in event class, subscription class, and notification class declarations to instruct the SQL-NS compiler to use a particular filegroup when creating the corresponding database objects. Listing 13.2 shows the use of the <FileGroup> element in the music store application's ADF.

Listing 13.2. Specifying Filegroups for the Event, Subscription, and Notification Classes

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <FileGroup>Primary</FileGroup>
      ...
    </EventClass>
  </EventClasses>

  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByArtist</SubscriptionClassName>
      <FileGroup>SecondaryGroup</FileGroup>
      ...
    </SubscriptionClass>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongByGenre</SubscriptionClassName>
      <FileGroup>SecondaryGroup</FileGroup>
      ...
    </SubscriptionClass>
  </SubscriptionClasses>

  <NotificationClasses>
    <NotificationClass>
      <NotificationClassName>NewSong</NotificationClassName>
      <FileGroup>Primary</FileGroup>
      ...
    </NotificationClass>
  </NotificationClasses>
  ...
</Application>

Each <FileGroup> element must specify the name of a filegroup in the application's database (the database specified in the ADF's <DatabaseName> element, not shown in Listing 13.2). This can either be the name of a filegroup declared in one of the <NamedFileGroup> elements, if the database is created by the SQL-NS compiler, or it can be the name of a previously created filegroup if the database was created earlier. In the case of the music store sample, the database is not created by the SQL-NS compiler; the filegroup names specified here are the names of the filegroups defined in the CREATE DATABASE statement in the script that sets up the music store database, as described in the sidebar "Creating Filegroups With T-SQL" (p. 463).

In Listing 13.2, the event class and notification class specify the Primary filegroup, and the subscription classes specify the SecondaryGroup filegroup. The events table, the notifications table, and other objects associated with the event and notification classes will be physically stored on the files associated with the Primary filegroup. The subscriptions tables and other related objects will be physically stored on the SecondaryGroup filegroup.

Caution

You can update the <FileGroup> element on an event, subscription, or notification class, but doing so causes the compiler to drop and re-create the associated database objects, including the tables. This means that any data in these tables is lost when you perform the update. Before updating an instance when you have changed the <FileGroup> values, be sure to back up any data in the tables you may need.

Choosing Appropriate File Sizes When Defining Filegroups

When specifying the size attributes for the data and log files used to store instance-level data, the most important consideration is the expected size and growth rate of the subscriber base. Because subscriber data (and subscriber device data) is stored in the instance database, specifying appropriate initial size and growth increment values for the instance database files will prevent SQL Server from having to perform costly file expansion operations as the subscriber base grows. Refer to the "Database Resource Planning" topic in the SQL-NS Books Online for information on calculating an appropriate size for your instance database.

Also, for best performance, it's important that the files in the filegroups you use for the event, subscription, and notification data are sized appropriately. When specifying initial size and growth increment values for the files, you must consider the expected size of the data and how quickly you expect it to grow. Again, the "Database Resource Planning" topic in the SQL-NS Books Online provides detailed guidance on calculating the size of the entities in the application database.

The size of the transaction log file is also important. The log file is used heavily during compilation operations, such as creating and updating the instance. Because both creating and updating typically involve many transacted operations, a significant amount of log space is required. The exact amount varies with the complexity of the instance and its applications, but for most applications, a minimum of 10MB of log space is recommended. If you set a maximum size on the log file, this should be quite large to ensure that create and update operations don't fail due to a lack of log space.

Setting the Database Recovery Model

You can specify the recovery model for a database to control the way that SQL Server logs the transactions performed against it. SQL Server supports three recovery models: simple, full, and bulk-logged.

When the simple recovery model is used, SQL Server does minimal logging for transactions executed against the database. This provides the best performance (because the logging overhead is small), but there is a potential for data loss in the case of a system crash or database corruption. SQL Server cannot always restore the database to an arbitrary point in time because complete logging information is not always kept.

In both the full and bulk-logged recovery models, SQL Server maintains detailed log information. Using this information, SQL Server can roll back or forward all previously executed transactions. The full and bulk-logged models differ in exactly when the log information is written and therefore make different guarantees about data loss. However, both models offer much stronger guarantees than the simple recovery model. This added reliability comes at the cost of increased logging overhead, which can bring down performance.

For most SQL-NS applications, the full recovery model should be used for the instance and application databases. Using the full recovery model minimizes the potential for data loss should a catastrophic failure occur. However, if you want the highest possible database performance and can tolerate some potential data loss, it may be appropriate for you to use the simple recovery model.

When a database is created, its recovery model is set to the recovery model of the SQL Server's model database. You can change the recovery model of a database after it is created using an ALTER DATABASE statement or by using Management Studio. See the "Selecting a Recovery Model" topic in SQL Server Books Online for more detailed information on the supported recovery models and the specific instructions for changing a database's recovery model.

Note

There are no dedicated ICF or ADF elements for specifying the recovery models for the instance and application databases.


Vacuuming Old Data

Up to this point, we have not really considered what happens to old event and notification data in a SQL-NS application's database after it has been processed. Generally, old data remains in the tables until it is vacuumed. Vacuuming is a process the SQL-NS engine runs periodically to delete old data that isn't needed anymore. By getting rid of unnecessary, old data, the SQL-NS engine can reclaim disk space and maintain the performance of the application.

In the ADF, you can configure a vacuuming schedule that specifies when the engine should clean up the application's old data. You can also specify the retention age for the old data. At the times specified in the vacuuming schedule, the engine runs the vacuum process, removing any items older than the specified retention age. Newer items are preserved (even if they have been completely processed).

Advantages Of The SQL-NS Vacuuming Model

The vacuum in the SQL-NS engine performs cleanup on a periodic schedule. You might wonder why this approach was chosen, instead of simply cleaning up event and notification data immediately after it is processed.

There are two reasons for the design choice. First, immediately removing processed data would make debugging, tracking, and auditing the behavior of a notification application difficult. Leaving old data in the tables, for some amount of time after it has been processed, provides a convenient history of the application's processing that can be queried by developers and administrators.

The other motivation for the chosen vacuuming strategy is that deleting data can adversely affect performance. Delete operations usually cause writes to disk and involve significant logging (depending on the recovery model used). If data was deleted as part of the normal processing pipeline, the throughput of the application would be reduced. Deleting data asynchronously, on a periodic schedule, has two performance advantages: The fixed costs of the delete operations can be amortized over larger sets of data, and the operations can be done at times of the day when the application is otherwise mostly inactive (during the middle of the night, for example).

This section describes the ADF elements used to configure vacuuming and provides some general guidelines for using vacuuming in SQL-NS applications. Although vacuuming falls into the category of maintenance tasks (covered in Chapter 14, "Administering a SQL-NS Instance"), it's important to think about vacuuming at the deployment stage because an application's vacuuming policy affects the amount of space needed on the database server's disks, as well as the appropriate size and growth settings for its database files.

Configuring Vacuuming in the ADF

A SQL-NS application's vacuuming configuration is specified in the <Vacuum> element of its ADF. Listing 13.3 shows a sample <Vacuum> declaration.

Listing 13.3. Configuring the Vacuum in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    ...
    <Vacuum>
      <RetentionAge>P3DT00H00M00S</RetentionAge>
      <VacuumSchedule>
        <Schedule>
          <StartTime>23:00:00</StartTime>
          <Duration>P0DT02H00M00S</Duration>
        </Schedule>
        <Schedule>
          <StartTime>03:00:00</StartTime>
          <Duration>P0DT02H00M00S</Duration>
        </Schedule>
      </VacuumSchedule>
    </Vacuum>
    ...
  </ApplicationExecutionSettings>
</Application>

The <Vacuum> element's two top-level subelements, <RetentionAge> and <VacuumSchedule>, specify the retention age for old data and the times at which the vacuum should run, respectively.

The <RetentionAge> element defines the minimum age for items to be vacuumed. The <RetentionAge> value is specified using the XSD duration syntax (described in detail in the sidebar "The XSD duration Data Type," p. 157, in Chapter 5, "Designing and Prototyping an Application"). In the example shown in Listing 13.3, the retention age is set to three days.

Note

The vacuum will not delete data that has not been completely processed, even if such data is older than the specified retention age. An example of old data that still requires processing is a set of notifications awaiting delivery retry (after having experienced delivery failures one or more times). The data associated with these notifications will be removed only after all delivery attempts have been completed.

The <VacuumSchedule> element specifies when the vacuum will run. The schedule consists of a series of <Schedule> elements, each of which specifies a start time and a duration.

Start times are specified in UTC, and durations are specified in the XSD duration syntax. For each vacuum schedule specified, the vacuum starts running at the given start time and continues either until the specified duration elapses or until all items eligible for vacuuming have been removed (whichever happens first). The <Duration> element is optional, and, if omitted, the duration is assumed to run from the given start time until the start of the next vacuum schedule. You can specify as many <Schedule> elements as you need to implement an appropriate vacuuming policy for your application.

Vacuum schedules have an implied recurrence of one day. In other words, the vacuum starts operating at the specified start times every day. If the <Duration> element is omitted from the last vacuum schedule declared, the assumed duration for that schedule runs from the given start time to the start time of the first vacuuming schedule on the next day.

Caution

Don't forget that schedule start times are specified in UTC. To ensure that the vacuum runs when you intend it to, covert local times to UTC before specifying them in the ADF.

Note

The SQL-NS engine runs only one vacuum process at a time. If you specify overlapping vacuum schedulesthat is, one schedule duration that runs past another schedule's start timethe vacuum adjusts the durations to make the intervals nonoverlapping.

Choosing Appropriate Vacuuming Settings

In any application, the appropriate retention age for old items is a trade-off between the need to retain historical data and the disk space and performance implications of storing that data. The appropriate retention age for your applications depends on their auditing and tracking requirements. You need to retain data for as long as you expect to have to answer detailed questions about notifications after they are generated. Some applications have relaxed auditing and tracking requirements, and, in these applications, a short retention age is appropriate. Other applications need to support customer queries about their notifications and therefore might need to keep several days' worth of old data.

The larger the retention age you use, the more disk space you will need for the application's database. You should ensure that your database system has sufficient capacity to store the data and that the file sizes are set so that file expansion does not have to be performed too frequently (see the "Defining the Physical Storage of SQL-NS Database Objects" section, p. 459, for details on specifying file size attributes). Also, be aware that as the tables in the database grow larger, queries against them become increasingly inefficient. This inefficiency can be mitigated, to some extent, by proper indexing.

Because the actual vacuuming operations can have a negative performance impact, it's important to plan the vacuum schedule carefully. During a given day, most SQL-NS applications have some periods of busy activity (around the times that events arrive and most scheduled subscriptions are set to be evaluated) and other periods of relative inactivity. Vacuuming should always be scheduled at the times that your application is expected to be least busy. You need to determine what these times are for your applications, based on predictions of the expected usage patterns, or, more reliably, based on observation of the application in production.

It's important not to underestimate the time required for vacuuming to complete. If, at the end of one run, the vacuum determines that it did not complete its jobthat is, it had to stop because the specified schedule duration had run out, but there were still remaining items to vacuumit will write a warning message to the Application Event Log. (For details on viewing messages in the Application Event Log, see the "Using the Application Event Log" section, p. 484, in Chapter 14.) If you notice this happening frequently, you should adjust the vacuuming schedule, either by adding more schedule elements or by increasing the durations of the existing ones.


Deployment Security

When designing, developing, and deploying a SQL-NS application, you should assume that at some time, a malicious user will try to find and exploit security vulnerabilities in it. Simply stated, your job is to prevent such a user from succeeding in this endeavor. You do this by eliminating security vulnerabilities wherever possible and configuring the system so that any potential vulnerability will provide only minimal access to the resources on the system.

This section describes some of the SQL-NS features that can be used to implement a secure deployment and outlines some security basics that apply to most deployment environments. The SQL-NS platform was designed with security in mind, but all the SQL-NS security features (as well as the security features of SQL Server and Windows) are just tools; they must be applied appropriately to be effective. The information presented here is intended only as a guide. Ultimately, the responsibility is yours to design and implement the appropriate security measures for your particular deployment environment.

Caution

When designing a security policy, it's easy to miss things, despite thinking carefully and methodically. Also, security threats and their recommended countermeasures are constantly evolving. Security policies should always undergo regular review and testing to ensure that they meet the demands of your environment.

Security Basics for the Deployment Environment

This section describes some of the recommended security practices for SQL-NS deployments. The information in this section builds on the security concepts for development environments covered earlier in this book.

Accounts and Authentication

As described in Chapter 2, "Getting Set Up," SQL-NS supports both Windows Authentication and SQL Server Authentication. Although both methods are supported, Windows Authentication is recommended because it is more secure and generally simplifies development and administration.

All running components of a SQL-NS instance (the SQL-NS engine, the SMI, and stand-alone event providers) run under the credentials of some Windows account. It's recommended that you use domain accounts to run these components and grant those accounts only the minimum permissions they need to carry out their functions. In the development environment, we used the same account to run all the components, but in deployment, it's a good idea to use different accounts for the different components. Doing this allows you to assign each account the smallest possible set of permissions. This minimizes the access that an unauthorized user will be able to gain, should any of the components be compromised.

All accounts (Windows accounts, as well as any SQL Server accounts used for SQL Server Authentication) should be configured with strong passwords. You can find guidelines for creating strong passwords on Microsoft's Trustworthy Computing website at http://www.microsoft.com/security/articles/password.asp.

Guarding Sensitive Information

In SQL-NS instances, sensitive information may exist in several places, including

  • Arguments to custom components specified in the ICF and ADFs

  • Code that uses SQL usernames and passwords to connect to the database using SQL Server Authentication

  • Environment variables used in scripts that invoke nscontrol commands

This section describes some techniques that can be used to guard this sensitive information from unauthorized access.

Arguments to custom components are read from the ICF and ADFs and are stored in instance and application database tables. Although the ICF and ADFs must themselves be secured (the next section provides information on this), the arguments in the database should also be protected using the argument encryption feature of SQL-NS. For detailed information on argument encryption, see the "Argument Encryption" section (p. 237) in Chapter 8, "Event Providers."

If you use SQL Server Authentication, you need to supply SQL usernames and passwords to various APIs in the implementation of SMIs and standalone event providers. For example, NSInstance objects require their SQL username and password properties to be set before they can connect to the instance database via SQL Server authentication. In the examples shown in this book, the SQL usernames and passwords were hard-coded in the source files. This was done for simplicity and ease of explanation but is not a secure practice. Anyone with access to the source code will be able to view the passwords, and anyone able to attach a debugger to the process running the code may be able to extract the passwords from the process's memory. Furthermore, hard-coding passwords presents difficulties for administrators because the code must be recompiled if the usernames or passwords change.

You should avoid hard-coding passwords in source files whenever possible. Alternatives are to store passwords in protected configuration files or Registry keys. But regardless of how secure the storage location may be, passwords should never be stored in plain text. Instead, they should always be encrypted using the Windows Data Protection API (see http://msdn.microsoft.com/library/en-us/dnsecure/html/windataprotection-dpapi.asp?frame=true for more information on this API). At runtime, components should read the encrypted passwords from the stored locations, decrypt them, and pass them to the APIs that require them.

Caution

It's especially important not to hard-code passwords in source files for web applications stored on web servers. Even with the appropriate security settings in place to deny web browsers access to the parts of the web server containing source code, the risk is not eliminated. Hackers are constantly finding security holes in web servers that allow them to view web application source code.

Scripts that invoke nscontrol commands may temporarily store sensitive information in environment variables. For example, in the scripts used with the samples in this book, the value for the -sqlpassword argument to nscontrol create is stored in and read from an environment variable. All this book's scripts clear out environment variables containing sensitive information before they exit. Sensitive information can remain behind in memory if a script does not clean up properly. In any scripts that you write, be careful to clear out any environment variables that contain passwords or other sensitive information. After you run the scripts, you should double-check that no sensitive information has been left behind in the environment variables.

Securing Source Code, Scripts, and Runtime Files

The files required to build a SQL-NS instance (the ICF, ADFs, source code for custom components, and build scripts) need to be carefully secured. As described in the previous section, they may contain sensitive information such as usernames and passwords, but they may also provide information about the internal structure of the instance and its components that a hacker might be able to exploit.

After the instance is created, the source files and build scripts don't need to reside on the deployment machines. The running instance executes solely against the information in the instance and application database objects, so the ICF and ADFs are not needed. Only files such as XSL transforms for the XsltFormatter and event schema files for the FileSystemWatcher are needed at runtime and do need to be on the deployment machines. These files should be locked down using Windows filesystem security and should be made accessible only to the accounts under which the components that need them run.

Note

Although they aren't needed on the deployment machine, the source files should always be kept in a safe, secure location on another machine. You will need these files to make changes to the application after it is created.

If your SQL-NS applications use custom components, the assemblies containing the implementations of those components need to be on the deployment machines as well. It's especially important to secure these assemblies. Because the SQL-NS engine loads the assemblies at runtime and executes the code they contain, if a malicious user is able to replace your assemblies with ones of his own, his code will run in the SQL-NS engine, under the credentials of the engine's account. Such code could be used to bring down your system, open it up for unauthorized access, or cause the SQL-NS engine to act on the malicious user's behalf.

You can secure custom component assemblies by signing them with a private key and referring to them in the ICF and ADFs with fully qualified strong names. In all this book's examples, the <AssemblyName> elements specified just a path to a DLL. However, every <AssemblyName> element in the ICF and ADF schemas also supports fully qualified strong names. A strong name qualifies the assembly name with an assembly version, culture, and public key token. For example, had our content formatter assembly been signed, it could have been specified in the <AssemblyName> element of a content formatter declaration as follows:

[View full width]

<AssemblyName>NewSongNotificationFormatter, Version=1.0.1234.0, Culture=en-US,
   PublicKeyToken=b77a5c561934e089d</AssemblyName>

Note

The version number and public key token shown in this snippet are only examples. The content formatter assembly actually used in this book's samples is not stamped with this version number nor signed to work with this public key token.

When you identify an assembly in an ICF or ADF with a strong name, the SQL-NS engine will not load it unless the file has been signed with the correct private key. This greatly reduces the chances that a trusted assembly could be replaced with malicious code.

For further security, you should also set the appropriate filesystem ACLs on your assemblies and run the SQL-NS engine under a low-privileged account. This ensures that even if a hacker is able to get the engine to run arbitrary code, that code would have very few permissions on the system.

Controlling Database Access

As you've already seen, access to SQL-NS databases is controlled via roles. SQL-NS defines a set of roles corresponding to the various functions of a running application. You can assign accounts to these roles as needed.

In the sample applications we looked at in previous chapters, all hosted components run on the same machine, so we made the single engine account a member of the NSRunService role. This role encompasses several other SQL-NS roles that each correspond to the individual functions of the components in the SQL-NS engine.

If you break out the components of the SQL-NS engine onto separate machines, as shown in Figure 13.4, you can run the engine under a different account on each machine. You can make the account used on a particular machine a member of only the roles needed for the components hosted on that machine. For example, the account under which the engine runs on the machine hosting event providers needs membership only in the NSEventProvider role. It does not need membership in the wider NSRunService role, which encompasses the permissions for the generator and distributor as well.

In a deployed SQL-NS instance, you can use different accounts for the SMI, the SQL-NS engine on each machine, standalone event providers, and administration and reporting operations. Each of these accounts can be assigned to the specific roles required. For a complete list of the SQL-NS database roles and the functions they correspond to, see the "Notification Services Database Roles" topic in the SQL-NS Books Online.

Using Throttles

For most SQL-NS applications, it's possible to predict the peak volumes of events, subscriptions, and notifications. Any of these volumes drastically exceeding their expected peaks might be an indication of a problem. The application might accidentally be processing more data than necessary due to a bug, or an intruder might somehow have induced the application into overload (perhaps in an attempt to flood the network with traffic). In either case, you probably want to curtail the application while you assess the cause of the deviation from the expected volumes.

SQL-NS provides a throttling mechanism that can automatically stop notification generation when specified limits are exceeded. Three throttle settings are provided:

  • An event throttle that limits the number of events that can be processed per event class in a given quantum

  • A subscription throttle that limits the number of subscriptions of each subscription class that can be processed in each rule firing

  • A notification throttle that limits the number of notifications that can be generated per notification class, as a result of each rule firing

You can specify these throttle values for an application in the ADF, as shown in Listing 13.4.

Listing 13.4. Configuring Throttle Values in the ADF

<Application>
  ...
  <ApplicationExecutionSettings>
    ...
    <EventThrottle>2000</EventThrottle>
    <SubscriptionThrottle>5000</SubscriptionThrottle>
    <NotificationThrottle>15000</NotificationThrottle>
  </ApplicationExecutionSettings>
</Application>

All three throttle values are specified as part of the <ApplicationExecutionSettings> element, in the <EventThrottle>, <SubscriptionThrottle>, and <NotificationThrottle> subelements, respectively. If the number of events, subscriptions, or notifications in a quantum exceeds the corresponding throttle value, the quantum's processing is halted. If this occurs, the generator writes a message to the Application Event Log, indicating that a throttle has been exceeded.

Note

Do not confuse throttles with the quantum limits described in Chapter 12, "Performance Tuning." Throttles limit the processing done in a single quantum, whereas the quantum limits constrain how many old quantums the generator processes when the quantum clock falls behind.

In your own applications, you should set the throttle values appropriately, based on the expected volumes of events, subscriptions, and notifications. After your application begins running, you should monitor the Application Event Log for messages indicating failures due to throttle limits being exceeded. If you see such failures, investigate whether there is a problem with the application, or whether the throttle values are too small. You can adjust the throttles as needed by editing the values in the ADF and then updating the instance.

The default value for all three throttles is 1,000. This value applies if the <EventThrottle>, <SubscriptionThrottle>, or <NotificationThrottle> elements are omitted from the ADF. If you specify a value of zero for any throttle, the throttle is effectively disabled, and no limit is imposed. Running with the throttles disabled is not recommended; even if you expect large volumes of data, you should still set throttle values, although they may need to be very high.

Caution

Because the throttle elements are all optional, it's easy to forget to adjust these defaults appropriately before deployment. In the development environment, applications are usually tested with small loads, so the throttle values aren't really noticed. However, in deployment, the default throttle value of 1,000, which applies if a throttle element is omitted from the ADF, is often exceeded in normal operation. Before you do large-scale testing or move your application into its deployment environment, make sure that you declare the throttle elements and supply appropriate values if you expect to exceed the defaults.


The Deployment Process

This section outlines the general procedure for deploying a SQL-NS instance. This is not intended to be a definitive, step-by-step set of instructions but rather a high-level walk-through of the stages in the deployment process. The exact sequence of steps you need to follow when deploying your SQL-NS instance depends on the deployment configuration you've chosen, the hardware you're using, the applications in your SQL-NS instance, and the components they employ.

Note

The information in this section assumes that the machines you will use in your deployment have already been configured with the appropriate hardware and operating system software. It also assumes that your SQL Server database engine is installed and ready for use.

Installing SQL-NS on the Deployment Machines

The section "Installing Notification Services and Other SQL Server 2005 Components" (p. 27) in Chapter 2 provides instructions on installing SQL Server 2005 components, including SQL-NS. You will need to install SQL-NS components on the machines in your deployment that host the SQL-NS engine or other programs that use the SQL-NS API (such as the SMI or standalone event providers). You use SQL Server 2005 setup to do this.

When you run SQL Server 2005 setup, you are given a choice of components to install. One of the options is Notification Services. If you choose this option, all the components of SQL-NS are installed. However, not every SQL-NS component is needed on every machine in your deployement, so a better strategy is to install components of SQL-NS selectively instead.

To do this, check the Notification Services option in the Components to Install dialog box, but then click the Advanced button before proceeding to the next step in the setup wizard. In the Feature Selection dialog box that opens, expand the Notification Services item in the feature tree. Beneath this item are two choices: Engine Components and Client Components. You can choose to install either one, or both.

As the name suggests, Engine Components includes everything required to run the SQL-NS engine. A SQL Server license is required for every machine on which the SQL-NS Engine Components are installed. The Client Components include only the features required to run SQL-NS client programs, such as SMIs and standalone event providers. A SQL Server license is not required for machines on which only the SQL-NS Client Components are installed.

Caution

Before you begin your deployment, make sure you have the required licenses to install SQL-NS on all your deployment machines. For more information on licensing, consult the SQL-NS licensing FAQ at http://www.microsoft.com/sql/technologies/notification/nslicensingfaq.mspx.

Tip

Instead of using SQL Server 2005 setup, you can use the SQL-NS Redistributable Package to install the SQL-NS Client Components. The Redistributable Package includes only the SQL-NS Client Components. If you need to install the SQL-NS Engine Components, you must use SQL Server 2005 setup. The SQL-NS Redistributable Package can be downloaded from http://go.microsoft.com/fwlink/?LinkId=54583 (search for "Microsoft SQL Server 2005 Notification Services Client Components").

In your deployment, you can selectively install SQL-NS components as follows:

  • Install the SQL-NS Engine Components on all machines on which the SQL-NS engine will run.

  • Install the SQL-NS Client Components on all machines that will host SMIs or stand-alone event providers.

However, if a single machine serves more than one function (such as hosting the SQL-NS engine and the SMI), you should install the components required for both functions on that machine. In the simplest single machine deployment, you can install all the SQL-NS components at once.

Specifying the Server Configuration in the ICF and ADFs

The ICF declares the name of the SQL Server to host the instance's databases, and the ADFs declare the machines on which the event providers, generator, and distributors will run. After you've decided on a particular server configuration for your deployment, you need to edit these declarations in the ICF and ADFs to ensure that they properly reflect the chosen configuration.

The SQL Server name is specified in the <SqlServerSystem> element of the ICF. The value specified should match the name of your deployment SQL Server. The host machines for the other components are specified via the <SystemName> element in the <HostedProvider>, <Generator>, and <Distributor> declarations in the ADFs. The <SystemName> values should reflect the names of the machines on which the components should run.

Note

On SQL-NS Standard Edition, specifying different <SystemName> values for different components in the same application results in a compile error. SQL-NS Standard Edition requires that all components run on the same machine, so the <SystemName> values must be the same.

Tip

<SystemName> elements in the ADF can specify either NetBIOS names or fully qualified domain names.

If you intend to run multiple distributors to scale out notification formatting and delivery (as shown in Figure 13.4), your ADF will need multiple <Distributor> declarations. In development, you probably used only a single distributor, so you may need to add new <Distributor> declarations to your ADF (one for each distributor in your deployment). The <SystemName> element in each <Distributor> declaration should specify the name of the appropriate host machine. Recall that you may configure only one distributor on any one machine.

Tip

It's a good idea to use parameters to specify machine names in the ICF and ADFs. This allows you to change the machine configuration without having to edit the declarations in these files. All the examples shown in this book specify machine names using parameters.

Deploying Custom Components

If your applications use custom components (custom event providers, content formatters, or delivery protocols), you need to copy the assemblies containing those components to the deployment machines on which they will be hosted. After you've copied the assemblies over, you will likely need to update the ICF and ADFs to ensure that the assembly locations are correctly specified.

The full paths to the custom event provider, content formatter, and delivery protocol assemblies are specified in the <AssemblyName> elements of their respective declarations in the ICF and ADF. In the development environment, these paths probably pointed to the directories in which the assemblies were originally built. After you move the assemblies to the deployment machines, you may need to change these paths to reflect their new locations.

Tip

If you use strong names for the assemblies, you may need to install them in the Global Assembly Cache (GAC) on the deployment machines.

Creating the Instance and Application Databases

The procedure for creating the instance and application databases in deployment is the same as in development. After you have your ICF and ADFs ready to compile, you invoke the SQL-NS compiler, either through Management Studio or nscontrol create. The instance and application database objects will be created in databases on the SQL Server identified in the <SqlServerSystem> element in the ICF (which should be your deployment server).

The creation process may take longer than it did in the development environment if the SQL-NS compiler needs to create databases and you specified large initial sizes for these databases' physical files. After the creation process completes, you should verify that the physical files have been created in the locations you expect.

Registering the Instance on All Deployment Servers

Registering an instance creates Registry keys used by the SQL-NS engine, and the SQL-NS API, to locate the instance and application database objects. In the process of registering, you can also install the SQL-NS Windows service and configure it to run under a specific account.

In your deployment environment, you need to register the instance on all machines that host the SQL-NS engine or programs, such as SMIs and standalone event providers, that use the SQL-NS API. You can register the instance using either the Register Instance command in Management Studio or the nscontrol register command-line tool. If you use the nscontrol register command, be sure to specify the name of the deployment SQL Server in the -server argument.

Caution

It's a common mistake to specify the local server name instead of the SQL Server name in the -server argument to nscontrol register. Always be sure that the -server argument specifies the name of your deployment SQL Server.

On machines that will run the SQL-NS Windows service, you should install the service when registering the instance. From Management Studio, you do this by checking the Create Windows Service box in the Register Instance dialog box. If you're using the nscontrol register command, you pass the -service argument to install the service for the instance. You need to specify the Windows username and password of the service account (and the SQL username and password if you are using SQL Server Authentication). Also, don't forget to specify the argument key if you are using argument encryption.

On the machines hosting only the SMI or standalone event providers (and not the SQL-NS Windows service), you can register the instance without installing the service. This creates just the required Registry keys and performance counters.

Granting Database Permissions to Deployment Accounts

All the accounts you use to run components in your SQL-NS deployment need permission to log in to your SQL Server and access the SQL-NS databases in the various SQL-NS roles. The procedure for granting the required database access is similar to that used in development, except that you may have to apply it to more accounts, and the roles that you use may be more specific. The "Granting Permissions" section (p. 92) in Chapter 4 provides details on the process of granting permissions.

Granting Filesystem Permissions to Deployment Accounts

If you use components that access files, you need to grant the appropriate filesystem permissions to the accounts under which those components run. The FileSystemWatcher event provider and the File delivery protocol are examples of components that require filesystem access. Although it's unlikely that you will use these components in a deployed SQL-NS application, you may have custom components that require filesystem access.

In keeping with the general security principle that accounts should be granted only the minimum required permissions, you should grant filesystem access selectively. Components should be allowed access only to the specific parts of the filesystem required for their operation. If an intruder is able to exploit a component to gain unauthorized access to your system, their access to the filesystem should be as limited as possible.

Deploying the Subscription Management Interface and Standalone Event Providers

SMIs and standalone event providers are really independent programs loosely coupled to the core SQL-NS application. Because SQL-NS does not place many restrictions on the nature and form of these programs, there are no SQL-NS specific instructions for deploying them. As long as the SQL-NS instance is registered on the machines hosting the SMI and standalone event providers, you can deploy them using whatever procedures you normally use to deploy independent programs in your production environment.

Enabling a Deployed Instance

After the instance has been created and registered and all required components deployed, you need to enable it before it can run. As in the development environment, you enable the instance using the nscontrol enable command or using Management Studio. The "Enabling the Instance" section (p. 95) in Chapter 4 provides details on the commands used to enable instances.

In the examples in this book, the enable commands are always used to enable the whole instance. However, you can also specify particular components of the instance to enable. This allows you to selectively enable only parts of the instance, while leaving others disabled. For more information on enabling or disabling specific components, see the "Enabling and Disabling Components" section (p. 492) in Chapter 14.

Starting the Instance

To start the instance, you need to start the SQL-NS engine on every deployment machine that hosts engine components. If the engine runs as a Windows service, you can use Management Studio to start the service on all the required machines. To do this, right-click the instance in the Notification Services folder of Management Studio's Object Explorer and choose Start from the context menu. Management Studio determines which machines host the service (from the machine configuration specified in the ADF) and starts the service on all those machines remotely.

Unfortunately, SQL-NS does not provide an equivalent command-line tool you can use to start the service on all the required machines. If you're working from the command line, you have to start the service on each machine manually, using the net start command.

If all the services start successfully, you know you have a running instance. Getting your SQL-NS instance running in the deployment environment for the first time is an important achievement! Although you may experience a few minor operational problems in the first days after the application is rolled out, seeing your applications in operation should be a satisfying experience.

Notes On Icf And Adf Code Samples In This Chapter

This chapter introduces several new ICF and ADF elements in the code fragment listings throughout the text. To see a complete ICF and ADF that use these new elements, look in this chapter's supplementary file directory, C:\SQL-NS\Chapters\13\SupplementaryFiles.

The ADF in this directory, ApplicationDefinition-18.xml, adds the code shown in Listings 13.2, 13.3, and 13.4 to the previous music store ADF. You can update or re-create the instance on your system with the code in this source file. For detailed instructions on re-creating the music store instance, see steps 49 in the section "Re-creating the SQL-NS Instance," p. 362, in Chapter 10, "Delivery Protocols." For instructions on updating an existing instance, see steps 27 in the "Testing the FileSystemWatcherProvider in the Music Store Application" section, p. 251, in Chapter 8. Be aware that if you update an existing instance, any data in the event, subscription, and notifications tables will be lost because filegroups are now specified for the event, subscription, and notification classes (as shown in Listing 13.2). This causes the SQL-NS compiler to drop and re-create the associated tables.

The ICF, InstanceConfiguration-4.xml, adds the <Database> element shown in Listing 13.1 to the previous music store ICF. This source file is included only for illustrationdo not use it to update or re-create your instance. As designed, the music store instance must be created in the preexisting music store database to be functional. The <Database> element in Listing 13.1 defines a new database.


Summary

After the application development phase is complete, a SQL-NS instance needs to be deployed on production hardware before it can be rolled out to users. This chapter covers the various aspects of the deployment process and describes the configuration options that SQL-NS supports. The choices you make when deploying a SQL-NS instance have a significant impact on its performance, reliability, and security.

SQL-NS supports several different machine configurations. In the simplest setup, all components of a SQL-NS instance, including the database, run on a single machine. For better performance and administrative flexibility, you can also separate the database from the SQL-NS engine by running them on different machines.

To provide increased processing power for running the application's core matching logic, you can scale up the SQL Server machine with additional processors, memory, and high-performance disk systems. You can also separate the components in the SQL-NS engine, as well as the SMI and standalone event providers, onto different machines. This allows you to provide them with dedicated hardware that can be tuned to the characteristics of their particular operations. To scale out the formatting and delivery work in the SQL-NS pipeline, an application can have multiple distributors, each running on a dedicated machine.

The database system is obviously a key part of any SQL-NS deployment. SQL Server allows you to control the way databases are mapped to physical files and to distribute those files optimally on the database machine's disks. For best performance, the data files, log files, and tempdb should be located on separate physical disks. The files should be created with sufficient capacity for the expected data volumes in the applications so that SQL Server does not have to perform costly file expansion operations often. This chapter describes the ADF elements used to specify the mapping of the SQL-NS database objects to physical files. In addition to physical storage, you can also specify a recovery model for the databases, which controls how much logging information is written for each transaction. This chapter provides a brief overview of the supported recovery models.

Over time, old event and notification data accumulates in SQL-NS application databases. SQL-NS removes old data by using a process called vacuuming. This chapter describes the ADF elements used to control vacuuming and provides guidance on how to choose an appropriate vacuuming schedule for your applications.

Security is critical in a deployed SQL-NS application. You should assume that your SQL-NS applications are potential targets for malicious users trying to gain unauthorized access to your systems. This chapter describes some basic security practices that can reduce the risk of a SQL-NS application being compromised and covers techniques for guarding sensitive data. The practices described here can minimize the potential damage that could result from a security breach. This chapter also describes event, subscription, and notification throttles, the features of SQL-NS that allow you to limit the volumes of data an application processes. Throttles can be an effective failsafe mechanism to alert you when an application's behavior deviates from expected patterns.

The final section of this chapter provides an overview of the deployment process. Deploying a SQL-NS application involves installing SQL-NS components on the required machines, updating the ICF and ADFs to reflect the chosen deployment configuration, deploying custom components, creating the instance and application databases, registering the instance, and granting database and filesystem permissions. Also, the SMI and any standalone event providers need to be deployed. When these deployment steps are complete, you can enable the instance and make it run by starting the SQL-NS engine on the required machines.

Using the information in this chapter, you can get your SQL-NS instance running in the deployment environment. Chapter 14 covers the administration of running instances, including how to monitor and maintain them.


Chapter 14. Administering a SQL-NS Instance

IN THIS CHAPTER

  • Monitoring and Reporting

  • Maintenance Techniques

  • Backup and Restore

This chapter explains some of the common tasks involved in administering a SQL-NS instance in a production environment. Administration tasks typically fall into two buckets: monitoring and maintenance. Monitoring tasks involve checking the instance periodically to verify that it is running as you expect. SQL-NS provides a number of facilities that you can use to monitor a running instance, including event logs, performance counters, and reports. Maintenance encompasses both the tasks you do regularly to keep the instance running smoothly and tasks that you do only when the need arisesto respond to problems, make configuration changes, or add functionality to your applications.

Before taking a SQL-NS deployment live, it's a good idea to practice some of the administration tasks described in this chapter. Although most SQL-NS applications tend to run trouble-free after a few initial problems are worked out, learning the proper administration techniques is still important. Careful administration will alert you to potential problems early and enable you to respond to them effectively.


Monitoring and Reporting

This section describes ways you can monitor a running SQL-NS instance and obtain reports about its activities. When monitoring, it's important to look out not only for explicit error messages and warnings from SQL-NS components, but also for deviations from expected levels of performance. For example, if the rate of event submission in an application suddenly drops, it might be an indication of a problem that requires your attention, even though you may not see an error message.

SQL-NS provides three main sources of information about the behavior of a running instance:

  • Error messages, warnings, and informational messages in the Application Event Log

  • A set of performance counters that provide specific performance data about individual SQL-NS components

  • A set of reports that provide overview information about the health of a whole instance, as well as detailed information about specific components and functional units

These information sources are described in detail in this section. You'll probably use all of them in practice.

Using the Application Event Log

Windows provides aset of standard event logs that services and applications (including SQL-NS) use to record events. The events in the event logs provide diagnostic information, error messages, and warnings. The Event Viewer tool (part of the Administrative tools on Windows operating systems) allows you to examine the event logs on your system.

Open the Event Viewer on a machine on which the SQL-NS engine runs. The left pane shows a list of event logs you can examine. You will see at least the three standard event logs listed: Application, Security, and System. Other event logs, created by applications or services on the machine, may also appear in the list.

The SQL-NS engine writes events to the Application Event Log. This is usually the first place you should look to assess the condition of a SQL-NS instance. Whenever a SQL-NS application in the instance exhibits unexpected or incorrect behavior, you should check the Application Event Log to see whether an error message has been logged.

When you click the Application log in the Event Viewer's left pane, the right pane shows a list of events. Double-click any event listed to see its properties. The Source column in the Event Viewer's right pane indicates which service or application logged each event. Events generated by the SQL-NS engine have a source value of NotificationServices. On your machine, there many not be any NotificationServices events if the SQL-NS engine did not encounter any errors.

Note

Entries in the event logs are generally called events but may also be referred to as messages. Be careful not to confuse the term events, when used in the context of the event log, with events in the SQL-NS sense (the input data submitted by event providers).

Entries in the event log are categorized as informational messages, warnings, or errors. These types are distinguished by the icon and label in the Type column. By default, the SQL-NS engine writes errors and warnings when it encounters problems in its execution. As described in the "Setting the Logging Level" section (p. 486), you can adjust the engine's configuration to make it log informational messages as well. These messages can provide detailed information about the behavior of the components in the engine, at various stages of processing. Although these messages can be useful, writing them involves a lot of overhead, so they are turned off by default.

SQL-NS Event Log Entries

When the SQL-NS engine encounters an error, or a situation that warrants a warning, the entry it writes to the event log contains a detailed description of the issue. Figure 14.1 shows an example of an error event generated by the SQL-NS engine.

Figure 14.1. The Event Viewer showing the properties of a SQL-NS event.

[View full size image]

Every event has an Event ID, an identifier that indicates the type of the event. For example, in NotificationServices events, an Event ID of 4045 indicates that the event represents a rule firing failure.

In NotificationServices events, the Description section of the Event Properties window always contains text that provides additional information about the occurrence that resulted in the event being logged. Usually, the description contains a specific error or warning message, along with the names of the instance, the application, and the component that generated the event. The descriptions in most NotificationServices events also contain a link to a web page that provides even more detailed information and troubleshooting assistance. Whenever you see an event log entry from NotificationServices for which you need more information, click the supplied link.

Note

The link pointing to additional information is usually at the end of the event description, so you may have to scroll down to see it.

This book does not provide a complete reference to all the SQL-NS events that you might see in the Application Event Log. Instead, Chapter 15, "Troubleshooting SQL-NS Applications," describes the common problem situations and how to diagnose them based on events in the Application Event Log.

Setting the Logging Level

You can adjust the SQL-NS engine's logging level to change the amount of information it writes to the event log. If you're having trouble diagnosing a problem, turning up the engine's logging level can provide additional information that might be helpful.

If the engine runs as a Windows service (as it usually does), you can adjust its logging level by editing the NSService.exe.config file that provides the service with its startup parameters. When the engine is hosted in a custom application process (instead of the service), you can control the logging level with settings in the hosting application's configuration file, as explained in Chapter 17, "Hosting the SQL-NS Execution Engine."

The NSService.exe.config file is always located in the same directory as the service executable, NSService.exe. You'll find this in the SQL-NS binaries directory, beneath the SQL Server install location. If SQL Server is installed in the C:\Program Files\Microsoft SQL Server directory, you'll find the SQL-NS binaries and associated files, including NSService.exe and NSService.exe.config, in C:\Program Files\Microsoft SQL Server\90\NotificationServices\9.0.242\bin.

NSService.exe.config is an XML document, and you can edit it with any XML or text editor. Although this file can be used to pass a number of configuration options to the SQL-NS service, it is most often used to specify logging levels by means of a set of log switches. Listing 14.1 shows the contents of the NSService.exe.config file as installed, before any changes are made.

Listing 14.1. Log Switch Settings in the NSService.exe.config File

<configuration>
    <system.diagnostics>
        <switches>
            <add name="LogAdministrative" value="2" />
            <add name="LogService" value="2" />
            <add name="LogEventProvider" value="2" />
            <add name="LogEventCollector" value="2" />

            <add name="LogGenerator" value="2" />
            <add name="LogDistributor" value="2" />
            <add name="LogVacuumer" value="2" />
            <add name="LogOther" value="2" />
            <add name="LogPerformanceMonitor" value="2" />
        </switches>
    </system.diagnostics>
</configuration>

Each log switch controls the logging for a particular component in the SQL-NS engine. For example, the LogGenerator switch controls generator logging. The value set for each switch determines the amount of information the corresponding component writes to the log. All switches are initially set to the value 2, which results in errors and warnings being logged. The log switches can be set to any value between 0 and 4, as follows:

  • 0No logging

  • 1Log only errors

  • 2Log errors and warnings

  • 3Log errors, warnings, and informational messages

  • 4Log errors, warnings, informational messages, and additional verbose informational messages

To obtain more or less information in the event log, you can adjust the log switch values for the relevant components. After changing the settings in the NSService.exe.config file, you must restart the service for the changes to take effect.

Caution

The log switch values that you set apply to the services for all SQL-NS instances on the machine. There is no way to control the logging levels independently for just one particular instance.

Note

Do not confuse the logging described in this section with the distributor logging described in the "Distributor Logging Options" section (p. 445) in Chapter 12, "Performance Tuning." This section describes log messages written to the Application Event Log by the SQL-NS engine (and the components in it). The distributor logging described in Chapter 12 refers to information the distributor writes to the database as it processes notifications.

SQL-NS Performance Counters

Registering a SQL-NS instance results in the installation of a set of performance counters you can use to obtain quantified data about the activities of the instance. A dedicated set of counters is installed for each instance registered. Monitoring the counters is an easy way to determine whether an instance is operating as it should: Sudden increases or decreases in counter values are easy to spot and can alert you to potential problems early.

SQL-NS performance counters are grouped into performance objects. Each performance object contains a set of related counters that measure various performance indicators related to a particular component or functional area. SQL-NS performance objects are divided into three categories:

  • Component-level performance objects for delivery channels, distributors, event providers, and the generator

  • Application-level performance objects for events, subscriptions, notifications, and the vacuum

  • An instance-level performance object for subscribers

For example, the events performance object (one of the application-level objects) contains counters that measure the following:

  • The number of event batches waiting to be processed by the generator

  • The number of event batches currently being submitted by event providers

  • The total number of event batches submitted since the application started

  • The total number of events received since the application started

Monitoring these counters gives you a picture of the event activity in the system. You can monitor performance counters using the built-in perfmon tool (part of Windows), a third-party monitoring tool, or programmatically, in monitoring applications of your own.

Note

More than one instance of some counters may be installed, based on the entities in the applications. For example, one instance of each event counter is installed for each event class.

The SQL-NS Books Online provides complete documentation for all the SQL-NS performance objects and the individual counters they contain. Consult the "Notification Services Performance Objects" topic in SQL-NS Books Online for a complete reference. The information presented in this book is intended to be a guide on how to use the performance counters and does not duplicate the detailed counter reference.

To see the SQL-NS performance counters in use, make sure you have the music store instance created, registered, and enabled, and then carry out the following instructions:

1. | If it is not already running, start the SQL-NS service for the music store instance. This is required because some of the performance counters are updated by code that runs in the service.
2. | On a machine on which you have registered the instance, start the perfmon tool: Open Administrative Tools and select Performance, or run perfmon.exe from a command prompt.
3. | The perfmon window contains two panes. The one on the left contains a tree view, and the one on the right contains a graph. Below the graph is a list of the counters currently being monitored. When you start perfmon, some system counters will be in the list, and their values will be reflected in the graph. Remove these counters from the list by selecting them and pressing the Delete key.
4. | Click the Add button above the graph (the Add button has an icon that looks like a plus sign). The Add Counters dialog box opens.
5. | The Add Counters dialog box is used to add to the list of counters being monitored. It allows you first to choose the computer from which to select counters. Select the local computer.
6. | In the Performance object drop-down list, you can choose the performance object from which to add counters. Expand the drop-down list to see the list of performance objects. The objects associated with a SQL-NS instance all have names that begin with NS$, followed by the instance name (for example, objects associated with the MusicStore instance have names that begin with NS$MusicStore). Select the events object for the music store instance, called NS$MusicStore: Events.
7. | The dialog box's list of counters will be populated with the counters in the selected performance object. Selecting a particular counter and then pressing the Explain button provides descriptive text for the counter. Select all the counters shown for the previously selected NS$MusicStore: Events object and click the Add button.
8. | Click the Close button in the Add Counters dialog box. Back in the main window, the events counters now appear in the list below the graph. Their values are represented by the lines on the graph.
9. | Execute the SubmitEvents.sql script (from the C:\SQL-NS\Samples\MusicStore directory) in Management Studio to add some events to the application.
10. | You should see some of the counter values change after you submit the events. Look at the Total Events counter, for example. There may be a significant delay (up to 1 minute) between the event submission and the counter updates, as explained later in this section.

Tip

Selecting an individual counter in the list causes its last, average, minimum, and maximum values to be displayed in the text boxes below the graph.

Figure 14.2 shows a screenshot of perfmon monitoring the events counters. If you carried out the preceding instructions on your system, you should see a similar display.

Figure 14.2. perfmon is used to monitor the event performance counters.

[View full size image]

The values of some performance counters are obtained by doing calculations over the data in the database. These calculations are performed by a performance-monitoring thread in the SQL-NS engine, on a periodic interval. By default, the performance monitor thread calculates and updates the counter values every 1 minute. Between these periodic updates, the counter values do not change. This is the reason for the delay mentioned in step 10 of the previous instructions.

You can adjust the interval on which the performance monitoring thread recalculates the counter values. To do this, use the <PerformanceQueryInterval> element in the ADF. (This element is not used in any of this book's examplessee the SQL-NS Books Online for information on this element.) Be aware that increasing the frequency of performance counter updates increases the load on the SQL Server and may therefore reduce the performance of your applications.

SQL-NS Reporting Stored Procedures

SQL-NS provides stored procedures that generate reports about instances, applications, and their subcomponents. The SQL-NS compiler creates these stored procedures in the instance and application schemas when an instance is compiled. You can invoke these stored procedures directly or from reporting tools to get information about the current state and history of your SQL-NS instance.

The reports produced by the SQL-NS reporting stored procedures fall into three categories:

  • Snapshot reports that provide information about the current state of the system

  • Diagnostic reports that provide troubleshooting information and details of failures

  • Analysis reports that provide historical and summary information about particular activities in the system

This section provides some examples of the various reports available. You should browse the "Notification Services Stored Procedures" topic in the SQL-NS Books Online to learn about all the other reports you can use. The SQL-NS Books Online provides detailed reference information for each report.

A Snapshot Report Example

To see an example of a snapshot report, you can invoke the NSSnapshotEvents stored procedure. This produces a report of the current event activity in an instance. For the music store instance, you can invoke it by running the following commands in Management Studio:

USE [MusicStore]
GO
EXEC [NSInstance].[NSSnapshotEvents]

Note

Like NSSnapshotEvents, many of the other reporting stored procedures are defined in the instance schema, even though they provide information about applications.

The resultset contains information about the enabled state of event submission and the time the last event batch was submitted for each event class in each application. This information allows you to quickly assess the state of the event submission activities. For example, an unusual increase in the reported elapsed time since the last event batch submission could be an indication of a problem with the event providers.

A Diagnostic Report Example

The diagnostic reports can help you troubleshoot problems by providing detailed information about the operation of individual components. For example, the NSDiagnosticEventClass stored procedure produces a report of event-processing activities for a particular event class. In the music store instance, it can be invoked as follows:

USE [MusicStore]
GO
EXEC [NSInstance].[NSDiagnosticEventClass] N'SongAlerts', N'SongAdded'

The arguments to NSDiagnosticEventClass specify an application name and an event class name. The stored procedure also takes other optional arguments that can be used to specify a time interval for which the report should be generated. The resultset returned summarizes the event batches submitted for the given event class and provides information on how they were processed.

Tip

In a running SQL-NS instance, it's better to use the reporting stored procedures to obtain information from the database rather than run queries directly against the tables. The reporting stored procedures are designed and implemented so as to avoid contention and locking problems while accessing tables used in the SQL-NS processing pipeline. Running your own queries directly against these tables can sometimes cause operations in the SQL-NS engine to time out or deadlock.

An Analysis Report Example

An example of a reporting stored procedure that produces an analysis report is NSQuantumPerformance. The data in this report summarizes execution times and other data related to processing of quantums in an application. NSQuantumPerformance is invoked from the application schema. To run it against the SongAlerts application, execute the following statements:

USE [MusicStore]
GO
EXEC [SongAlerts].[NSQuantumPerformance]

The resultset shows data about the quantums processed by the application. This gives you an indication of how well the generator is keeping up with its workload.


Maintenance Techniques

A SQL-NS instance will probably undergo a number of changes during its lifetime. Operational problems may require adjustments to configuration settings or the applications' code. External systems (on which the applications depend) may evolve. Deployment hardware is sometimes replaced or upgraded. The applications themselves might need to change in response to new requirements or feature requests. Maintenance is the art of keeping the instance running in the face of such changes, allowing it to grow and evolve over time.

Maintenance also involves some tasks that need to be done regularly to ensure smooth operation. For example, old data in chronicle tables may need to be removed to free up disk space and improve query performance. This section describes the basic techniques you use when maintaining SQL-NS instances.

Enabling and Disabling Components

As you saw in previous chapters, a SQL-NS instance can be enabled and disabled via Management Studio or the nscontrol enable and nscontrol disable commands. In most of this book's examples, we used these tools to enable or disable a whole instance at once. However, both Management Studio and the nscontrol commands also provide the capability of enabling or disabling individual components within an instance or application. This can be useful when maintenance tasks require one component to be taken down (for repairs or changes to be made) while leaving others running.

Enabling and Disabling Components with Management Studio

In Management Studio, the components of an instance can be enabled and disabled from the instance's properties dialog box. To open the instance properties dialog box, right-click an instance in the Notification Services folder in the Object Explorer tree and choose Properties.

The Instance Properties dialog box has four pages. You select a page to view using the page selector in the pane on the dialog box's left side. The Applications page provides controls for enabling and disabling components of an application, and the Subscribers page has controls for enabling and disabling subscriber management for the whole instance.

In the Applications page of the Instance Properties dialog box, the first drop-down list allows you to select among the applications in the instance. When you select an application, the components grid beneath the drop-down list is populated with that application's components.

To enable a component, check its box in the Enable column (the first column in the grid). To disable a component, uncheck its box in the Enable column. Note that the first component, labeled Application, displayed in the components grid represents the application as a whole. Checking or unchecking the Enable box in the Application component's row is a convenient way to enable or disable all components in the application. This is also the only way to enable an application's vacuum, which is not listed as a separate component.

To enable or disable subscriber management for an instance, go to the Subscribers page of the Instance Properties dialog box. The Subscribers page shows the current state of subscriber management and provides a check box you can use to change it.

Click the OK button in the Instance Properties dialog box to apply any changes made to the component states. These changes take effect as described in the section "Applying Component State Changes" (p. 495).

Enabling and Disabling Components with nscontrol

The command-line help for nscontrol enable shows that it takes the following arguments:

Usage: enable [-nologo] [-help]
    [<component>] -name <instanceName>
    [-server <databaseServer>] [-application <applicationName>]
    [-sqlusername <sqlUserName> -sqlpassword <sqlPassword>]
    where <component> can be one or more of the following:
        -events [<systemName>]
        -generator
        -distributor [<systemName>]
        -subscriptions
        -subscribers

The -name argument (or -n for short, as we have used it in this book's examples) is always required to specify the instance name. Specifying only this argument causes the whole instance to be enabled. You can restrict the scope of the enable command by using additional arguments.

The <component> argument can be used to specify a particular component (or functional area) to enable. For example, you can enable the generator by supplying the -generator argument, or you can enable the subscription management functions by supplying the -subscriptions argument.

The -events and -distributor arguments each take an optional <systemName> qualifier. This is used to restrict the components affected by the command to just those configured to run on a particular machine. For example, to enable only the distributor on a machine called DistributionServer1, you would specify -distributor DistributionServer1. Similarly, when the -events argument is used, supplying a <systemName> qualifier makes the command apply only to hosted event providers on the given system. When the <systemName> qualifier is omitted, the command is applied to hosted event providers on all systems.

Caution

In their ADF declarations, standalone event providers are not associated with any particular machine. When nscontrol enable is invoked with the -events argument, standalone event providers are always enabled, whether or not a <systemName> qualifier is used (and regardless of the value given).

The -application argument is used to restrict the components affected by the command to those in a particular application within the specified instance. The -application argument can be used in conjunction with the other arguments that identify application subcomponents, or it can be used alone. For example, specifying -application SongAlerts -generator makes the command enable only the generator in the SongAlerts application (leaving the state of the generators in all other applications in the instance unmodified). When used without another argument to restrict the command to an application subcomponent, the -application argument causes the command to apply to all components in the application. Omitting the -application argument causes the command to apply to all the applications in the instance.

Tip

Unfortunately, there isn't a -vacuum argument that can be used to specifically apply the enable command to the vacuum component. Using the -application argument, without any other subcomponent arguments, is the only way to change the state of the vacuum in an application.

Note

The arguments to nscontrol disable are the same as those listed for nscontrol enable, and they work the same way. Using arguments as described in this section, you can restrict the scope of the disable command to particular components or applications.

Applying Component State Changes

It's important to understand that the enabling and disabling components with Management Studio or the nscontrol commands does not manipulate the running components in the SQL-NS engine directly. These commands set flags in various instance and application database tables to indicate that the state of the components should change. The SQL-NS engine periodically polls these flags, and when it sees changes, it responds by enabling or disabling the relevant components. Because of this polling scheme, there may be a delay between the time that you issue an enable or disable command and when the engine actually implements the changes to the states of the running components. Usually, this delay is no more than 30 seconds.

The only exceptions are the enabling and disabling of subscribers, subscriptions, and standalone event providers. For these, state changes take effect immediately because the state flags in the database are checked by the subscription management and event submission APIs on every invocation.

Note

If the SQL-NS engine is not running, changes to component states made via Management Studio, nscontrol enable, or nscontrol disable will affect which hosted components will run the next time the engine starts.

Cleaning Up Old Data

As described in the "Vacuuming Old Data" section (p. 467) in Chapter 13, "Deploying a SQL-NS Instance," the data that accumulates in SQL-NS application database tables must be cleaned out regularly. The SQL-NS vacuum can be configured to run automatically, on a configurable schedule, to remove the data related to old events and notifications.

This section covers two additional aspects of the cleanup problem: invoking vacuuming manually and cleaning out data that is outside the scope of the vacuum. Using the techniques explained in this section to perform complete and regular cleanup is a critical part of application maintenance.

Invoking Vacuuming Manually

SQL-NS allows you to invoke the vacuum manually, if you need to remove old data at a time when the vacuum is not normally scheduled to run. You might need to do this if the scheduled vacuuming runs fail to remove all the old data (because the scheduled intervals are too short or because of errors during vacuum execution).

To invoke vacuuming manually, you execute the NSVacuum stored procedure, which the compiler creates in the application schema. NSVacuum takes one required argument: the maximum number of seconds it should run. This argument allows you to specify a time limit for the vacuum's execution. NSVacuum uses the same logic that the regular, scheduled vacuum process uses: It removes all processed data that is older than the application's configured retention age. NSVacuum runs until the specified time limit is exceeded or until there are no remaining items to vacuum, whichever happens sooner.

Note

The argument passed to NSVacuum is often mistaken for a retention age. Remember that the NSVacuum argument specifies only the number of seconds that the vacuuming process should run, not the age of the items it should vacuum. The retention age specified in the <RetentionAge> element in the ADF still applies, even when vacuuming is invoked manually. To change the retention age, edit the <RetentionAge> element in the ADF and then update the instance.

In the music store database, you can invoke NSVacuum as follows:

USE [MusicStore]
GO
EXEC [SongAlerts].[NSVacuum] 600

The argument in this example specifies a maximum run time of 600 seconds, or 10 minutes. If you run this code snippet against the music store database on your system, NSVacuum will most likely return immediately because the database in its current state probably doesn't contain any items older than the retention age to vacuum.

NSVacuum returns a resultset containing a single row with three columns:

  • StatusIndicates the status of the vacuuming operation. A value of 2 indicates that vacuuming completed successfully, and a value of 3 indicates that the time limit was exceeded while there were still remaining items to vacuum.

  • QuantumsVacuumedIndicates the number of quantums for which data was removed.

  • QuantumsRemainingIndicates the number of quantums eligible for vacuuming that were not vacuumed because NSVacuum ran out of time.

Note

The vacuuming process is based on quantums, which is why the amount of data vacuumed and the amount of data remaining are both expressed as a number of quantums. The vacuum considers quantums older than the retention age (as measured by the quantum end time) and marks them as eligible for vacuuming if all the event data submitted during the quantum and all notification data generated during the quantum has been completely processed. It then proceeds down the list of marked quantums and deletes the event, notification, and other associated data.

Cleaning Up Chronicle Tables

The data in chronicle tables is outside the scope of the SQL-NS vacuum. SQL-NS does not control the schema or semantics of chronicle tables, so it cannot manage their cleanup automatically. When the vacuum runs, the chronicle tables are not affected.

If your application accumulates data in chronicle tables, you must implement your own cleanup mechanism to remove old data that is no longer needed. The retention policy you use to determine when data can be cleaned up depends on the semantics of the chronicle table and the purpose it serves in your application's logic.

For example, an event chronicle table that keeps event history will probably be cleaned up differently from a subscription chronicle that keeps state for each subscription. The event chronicle can be cleaned up based on the age of the event data, but rows should be removed from the subscription chronicle only when the corresponding subscriptions are deleted.

You should write a SQL cleanup script for each chronicle table in your application. In each script, you can implement the cleanup policy appropriate for the particular chronicle table. You can run these cleanup scripts manually at any time, or you can set up SQL Server Agent jobs to run them automatically on a regular basis. For more information on using SQL Server Agent to set up scheduled tasks, consult the SQL Server Books Online.

Tip

It's a good idea to create a separate cleanup script for each chronicle table rather than lump the cleanup logic for multiple chronicle tables into one script. Using separate scripts allows you to easily clean up chronicles selectively, one at a time.

This chapter's scripts directory, C:\SQL-NS\Chapters\14\Scripts, contains SQL scripts that clean up the two chronicle tables defined in the music store sample application. The script CleanupSongAddedLog.sql removes old data from the SongAddedLog event chronicle table, and the script CleanupNewSongByGenreSubscriptionProcessingTimes.sql removes data from the NewSongByGenreSubscriptionProcessingTimes subscription chronicle. These scripts can be invoked manually or called from SQL Server Agent jobs.

Note

For a review of the music store application's chronicle tables and their semantics, see the "Event Chronicles" section (p. 163) and the "Subscription State" section (p. 170) in Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State."

Listing 14.2 shows the code in the CleanupSongAddedLog.sql script.

Listing 14.2. Cleanup Script for the SongAddedLog Chronicle Table

USE [MusicStore]
GO

DECLARE @oldestProcessingTime DATETIME
DECLARE @oldestCreationTime DATETIME
DECLARE @cutoffTime DATETIME

--Determine the oldest processing time in the subscription chronicle.
SELECT @oldestProcessingTime=MIN(LastProcessingTime)
FROM   [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]

--Determine the creation time of the oldest subscription not in the
--subscription chronicle.
SELECT  @oldestCreationTime=MIN(subscriptions.Created)
FROM    [SongAlerts].[NSNewSongByGenreSubscriptions] subscriptions
WHERE   NOT EXISTS (
    SELECT chron.SubscriptionId
    FROM   [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes] chron
    WHERE  chron.SubscriptionId = subscriptions.SubscriptionId
)

--The cutoff time is the earlier of the oldest processing time and
--the oldest creation time.
IF @oldestProcessingTime < @oldestCreationTime
    SELECT @cutoffTime = @oldestProcessingTime
ELSE
    SELECT @cutoffTime = @oldestCreationTime

--Delete items from the event chronicle older than the cutoff time.
DELETE FROM [SongAlerts].[SongAddedLog]
WHERE TimeAdded <= @cutoffTime

Remember that the event chronicle's purpose is to provide an event history against which the scheduled subscriptions are evaluated. We cannot simply delete rows from the event chronicle based on a fixed retention age because some of those rows may still be needed for scheduled subscriptions that have not yet been evaluated. The cleanup script works by using the subscription chronicle and the subscription data to determine a cutoff time for the events that should be left in the event chronicle. Events older than the cutoff time are deleted, and the rest are left behind. As long as the cutoff time is calculated correctly, we can be sure that events that are still needed in the event history do not get deleted.

The cutoff time is determined by looking at the oldest processing time recorded in the subscription chronicle and the creation time of the oldest subscription that has not yet been evaluated. The earlier of these two times becomes the cutoff. This works because of the way the scheduled subscription-matching logic scopes the data in the event chronicle. All subscriptions that have been evaluated at least once have already seen events older than their processing times recorded in the subscription chronicle. The oldest processing time in the subscription chronicle gives the cutoff time for the oldest event that all evaluated subscriptions have seen.

Subscriptions that have not yet been evaluated will never be evaluated against events older than their creation times (because the last processing time in the subscription chronicle is initialized to the creation time of the subscription on first evaluation). Thus, the creation time of the oldest subscription without an entry in the subscription chronicle provides the cutoff time for events that will never be seen by the yet unprocessed subscriptions. The earlier of the two cutoff times determines which events can safely be deleted.

Caution

Although a strategy like this guarantees that the required event history will be kept for all scheduled subscriptions, it may not clean up the chronicle fast enough from a maintenance perspective. A single scheduled subscription with an infrequent schedule can cause data to be left in the chronicle for extended periods of time. For example, a scheduled subscription due to be next evaluated six months into the future would require all chronicle data that arrived after its creation time to remain in the chronicle until it is evaluated.

A compromise would be to declare a maximum age for chronicle data and have the cleanup script remove all data older than this age, regardless of whether any outstanding scheduled subscriptions will ever be evaluated against it. By using a relatively large maximum age, you can handle the vast majority of subscriptions correctly.

Listing 14.3 shows the code in the CleanupNewSongByGenreSubscriptionProcessingTimes.sql script, which cleans up the subscription chronicle.

Listing 14.3. Cleanup Script for the NewSongByGenreSubscriptionProcessingTimes Chronicle Table

USE [MusicStore]
GO
DELETE FROM [SongAlerts].[NewSongByGenreSubscriptionProcessingTimes]
WHERE SubscriptionId NOT IN (
    SELECT SubscriptionId
    FROM   [SongAlerts].[NSNewSongByGenreView]
)

This script is much simpler than the one used to clean up the event chronicle. It simply deletes rows from the subscription chronicle that do not have corresponding rows in the view of all subscriptions. Such rows correspond to subscriptions that were evaluated at least once but were since deleted.

Note

As an alternative to the asynchronous cleanup strategy described here, the subscription chronicle could be maintained by the SMI's logic. Whenever a subscription is removed, the corresponding chronicle entries could be removed, too.

Making Changes to Deployed SQL-NS Instances

To make changes to a deployed SQL-NS instance, you use essentially the same techniques that you used when you made incremental changes during development. First, you edit the ICF or ADFs and make the necessary modifications. Then you disable the instance and run either nscontrol update or the Update Instance command in Management Studio.

Although the procedure is the same as in the development stage, you must be a lot more careful when performing updates against a deployed instance. Because SQL-NS requires an instance to be disabled before it can be updated, you must schedule the update so that the downtime has the least impact on users. While an instance is disabled, users will not be able to enter subscriptions, event providers will not be able to submit events, and no notifications will be generated or delivered. During the downtime, you may want to take the SMI offline to prevent users from trying to add or manage subscriptions.

In addition to scheduling the update properly, you also need to be concerned with potential data loss and corruption of the instance. The update process is atomic and the SQL-NS compiler will always leave the instance in a consistent state after it completes an update, but problems can still occur. For example, some changes introduced in an update are difficult to reverse. If these changes don't have the effects you intended, you may be left with an inoperable instance. Also, some updates rebuild tables, causing existing data in them to be lost. This section describes the steps you should take, before and after performing an update, to protect your instance against data loss and other problems.

Note

Despite the warnings about the potential pitfalls of the update process, the SQL-NS update tools (nscontrol update and the Update Instance command in Management Studio) are still the only means you should ever use to make changes to your instance. You should never update the instance or application database objects directly, even for seemingly trivial changes. The warnings given here are intended only to alert you to potential dangers so that you can take the appropriate precautionary measures.

Tip

To minimize the chances of problems, you should do updates incrementally. Instead of making several changes to an ADF at once and then performing a single, large update operation, it's better to make the changes individually and use several update operations to apply them one at a time. The more complex the changes introduced in an update, the greater the chance that something could go wrong.

Preparing to Perform an Update

The safest approach to performing an update is to first do a full backup of the databases containing the instance and application objects. This allows you to restore them to their original states should a bad update leave them in an unrecoverable condition. The section "Backing Up a SQL-NS Instance" (p. 502) later in this chapter provides information on how to perform backups.

Performing an update (even if it completes successfully) may result in data loss in the following cases:

  • If the changes in the ADF require the compiler to rebuild the event class or notification class tables, data in those tables will be lost after the update completes. Changing the schemas of these classes, including the <IndexSqlSchema> elements, usually causes these tables to be rebuilt.

  • If the compiler detects changes to the chronicle statements, they will be rerun. If these statements are written so that they drop and re-create existing tables, any data in the tables will be lost. For an example of such chronicle statements, see Listing 6.1 (p. 164) in Chapter 6.

  • If you make changes to the subscription class schema declarations (including <IndexSqlSchema> elements), the subscription tables will be rebuilt. Before rebuilding the tables, the compiler renames the existing tables to preserve the data in them (this mechanism is described in the "Testing the Effects of the New Indexes" section, p. 432, in Chapter 12).

If you will need any of the data that will be lost during the update, be sure to make a copy of it before performing the update.

Performing an Update

After backing up the required data, invoke nscontrol update or the Update Instance command in Management Studio to compile your ICF and ADF changes. For details on both commands, see the section "Making Updates to an Instance and Its Applications," (p. 131) in Chapter 5, "Designing and Prototyping an Application." After examining the modified ICF and ADFs, the update commands display the list of changes they will make to the databases and then ask for your confirmation before proceeding. Although the displayed list of changes is sometimes difficult to read, it's worth scanning to ensure that the update will do what you expect.

If the update tools encounter an error while applying changes to the databases, they roll back all changes made up to that point. This leaves the databases in their original state. Usually, you can fix whatever part of the ICF or ADF code resulted in the error and run the update command again. You may need to repeat this process several times until the update completes successfully.

Post-Update Steps

After an update has completed successfully, you may need to take additional steps before you can get the instance running again. Specifically, you must restore any lost data you require, either from the database backup or from other copies that you made.

If your changes resulted in subscription tables being recompiled, make sure that you copy any old subscription data from the old renamed subscription tables into the new ones. It's particularly important to copy back the subscription schedule data in the case of scheduled subscriptions. The section "Testing the Effects of the New Indexes" (p. 432) in Chapter 12 explains this process in detail and includes information on a sample script you can look at for an example of how subscription and schedule data should be restored.

After copying the old data into the new tables, you should drop the old tables. If you leave these tables around, any subsequent updates that require renaming of the subscription tables will fail because the update tools will try to reuse the existing old table names, generating a conflict.

Finally, after restoring all data, it's important that you test the instance before making it available to users again. Just because the update command succeeded, there is no guarantee that the changes will work as you expect. It's always important to verify, in the production environment, any changes you make.


Backup and Restore

This section explains how to perform a backup of a SQL-NS instance, as well as how to restore an instance from a previous backup. Doing regular backups is an important part of administering an instance and is the only way to ensure that it can be recovered after a catastrophic failure.

Backing Up a SQL-NS Instance

Backing up a SQL-NS instance involves backing up the following individual items:

  • The databases containing the instance and application objects

  • The runtime files (such as XSL transforms) required by application components, the assemblies containing custom components, standalone event providers, and any SMI files

  • The registration information for the instance, including the accounts used to run the various components

  • The source files, including the ICF, ADFs, and code for any custom components

To back up the databases, you can use any of the standard SQL Server backup techniques. SQL Server provides built-in commands for backing up entire databases, as well as specific files or filegroups. See the BACKUP command in the SQL Server Books Online for more information. You can also use Management Studio or a number of third-party tools to back up SQL databases. Any of these can be applied to the SQL-NS databases, just as they can to other databases.

Tip

If the instance and application database objects are created in different databases, make sure you back up all databases on the same schedule so that the backups are consistent with one another.

When backing up the runtime files, custom component assemblies, and other files, it's important to preserve not only the files themselves but also the directory structure in which they are laid out. This is especially important for the XSL transforms used with the XsltFormatter. The XsltFormatter relies on the directory layout to locate the appropriate transform for each locale and device type (see the "Using Locale and Device-Specific Transforms" section, p. 329, in Chapter 9, "Content Formatters," for more information). You can use any of the standard file backup tools to fully back up all directories and files used by the SQL-NS instance.

It's highly recommended that you preserve a record of the registration information for the instance (for all machines in the deployment) as well as the accounts used to run all the components. When restoring an instance from a backup, you will need to reregister it, and you may need to grant database and file permissions to the accounts again. Having a record of this information makes this process much easier. How you keep this record is up to you; in most cases, you can store it in a document, along with the instance's source files. If the document describes passwords or other sensitive information, make sure that it is secured appropriately.

The source files that make up the definition of an instance should be maintained in a version control system and backed up regularly. Even if you are able to restore a running instance, losing the source files will make it difficult to update the instance in the future. Using a version control system allows you to track changes to your source files as they evolve.

Tip

You can re-create the ICF and ADFs for a compiled instance using the nscontrol export command or the Export to Editor task in Management Studio. (See the SQL-NS Books Online for information on using these tools.) These operations reconstruct the original XML definitions from the compiled information in the instance and application database objects. Although these export tools exist, you should still keep a copy of the original source files in a safe, secure location.

Restoring a SQL-NS Instance from Backup

Before you can restore an instance from a previous backup, you need to restore the deployment hardware on which it will run. In the case of hardware failure, this may mean rebuilding some of the machines and reinstalling the required software on them. The "Installing SQL-NS on the Deployment Machines" section (p. 476) in Chapter 13 provides guidance on installing SQL-NS on deployment hardware; you can refer back to this section if you need to rebuild your SQL-NS machines.

After the deployment machines have been restored, you can use the following instructions to restore the SQL-NS instance:

1. | Restore the instance and application databases from their backups, using standard SQL Server procedures. See the RESTORE command in the SQL Server Books Online for more information.
2. | Rebuild the instance metadata. To do this, run the nscontrol repair command. The general topic of instance metadata is covered in the "Instance Metadata" section (p. 100) in Chapter 4, "Working with SQL-NS Instances." However, the nscontrol repair command is not covered in this book. For documentation, see the SQL-NS Books Online.
3. | Rebuild the directories containing the instance's runtime files and custom component assemblies. Take care to restore the directory structure as it existed before.
4. | Restore any standalone event providers and the SMI if necessary.
5. | Re-create any accounts required to run the components of the instance.
6. | Reregister the instance and grant database and filesystem permissions as needed.
7. | If you are restoring the instance onto different machines, you may need to update the <SqlServerSystem> and <SystemName> elements in the ICF and ADFs accordingly. Also, if any paths have changed as a result of restoring the instance's runtime files in step 2, you may need to update the paths specified in the ICF and ADFs. After making the required ICF and ADF changes, update the instance to reflect them in the databases.
8. | Enable the instance.
9. | Restart the instance by starting the SQL-NS engine on all machines hosting it.

As part of deploying your instance, I recommend that you practice restoring it from backup at least once. Doing this familiarizes you with the process and helps you find and fix problems in your backup strategy. Practicing in advance enables you to recover your systems much more quickly should the real need arise.


Summary

This chapter provides a description of the typical administration tasks applied to SQL-NS instances. Administering a SQL-NS instance involves both monitoring it for problems and maintaining it in the face of changes.

SQL-NS provides three main information sources you can monitor to keep track of the health of an instance: entries in the Application Event Log, a set of performance counters, and database reports. This chapter explains the information you can obtain from these sources and the tools used to access them.

Maintaining an instance involves tasks that you do regularly to ensure trouble-free operation and tasks that you do to implement required changes. This chapter describes cleanup tasks that must be done regularly and techniques for making changes to applications as needed.

Another important part of administering an instance is performing regular backups. This chapter provides information on backing up the parts of a SQL-NS instance and restoring them again to an operational state.

The techniques and tools described in this chapter will help you keep your instance running efficiently, reliably, and securely in its deployment environment. Chapter 15 builds on this material by providing a guide to troubleshooting common problems that you might encounter.


Chapter 15. Troubleshooting SQL-NS Applications

IN THIS CHAPTER

  • Using the Troubleshooting Information in This Chapter

  • Troubleshooting Compilation

  • Troubleshooting Event Submission

  • Troubleshooting Compilation

  • Using the Troubleshooting Information in This Chapter

This chapter is a general troubleshooting guide for problems that commonly occur while building and running SQL-NS applications. This is not intended to be a chapter that you read from start to finish, but rather one in which you consult specific sections to find solutions to SQL-NS problems.


Using the Troubleshooting Information in This Chapter

The information in this chapter consists of a set of problem descriptions and suggested solutions. For each problem covered, a short description of the observable symptoms is given, followed by an explanation of the root cause and a suggested course of action.

The sections of this chapter are organized first by functional area and then by problem symptoms. For example, one section is devoted to problems you might encounter while compiling an application, and another covers problems that can occur during event submission (in a running application). Within the compilation section, there are separate subsections for problems that manifest themselves as syntax errors and those that appear to be SQL connectivity issues.

When you encounter a problem while working with SQL-NS, find the section of this chapter that corresponds to the functional area you're dealing with. Then look for a subsection that matches the specific symptoms you see. In some cases, there may not be an exact match, but reading problem descriptions that seem related may give you clues as to how to solve, or further diagnose, the problem you are experiencing.

When you compile or administer a SQL-NS application, problem symptoms are generally easy to observe because the tools provide immediate feedback. In running a SQL-NS application, symptoms are sometimes more difficult to see because the SQL-NS engine does not provide direct feedback. To see information about problems that the engine encounters, you need to consult the Application Event Log on your system. Refer to the "Using the Application Event Log" section (p. 484) in Chapter 14, "Administering a SQL-NS Instance," for information on how to obtain error messages, warnings, and informational messages from the Application Event Log.

Note

The information in this chapter can be used to troubleshoot problems you may experience with the sample applications in this book, as well as other applications of your own. If you are having trouble with the sample applications, you can also look for troubleshooting information and source code updates in the errata section for this book on the Sams Publishing website. For instructions on how to access this information, see the note in the "About the CD-ROM" section (p. 4) in this book's Introduction.


Troubleshooting Compilation

This section covers problems you might encounter while compiling a SQL-NS instance for the first time or when updating an instance after it has been created. The guidance given in this section is the same, regardless of whether you use the command-line nscontrol create and nscontrol update commands, or Management Studio's New Notification Services Instance and Update Instance commands, to compile your instance.

Problems with ICFs and ADFs

This section covers compilation problems that result from errors in ICFs and ADFs.

Note

Error messages are shown in this section exactly as they are displayed by the nscontrol create and nscontrol update command-line tools. If, instead of these command line-tools, you are using Management Studio to compile your instance, and you encounter the errors described in this section, the error messages you see may differ slightly in format from the ones shown here. However, the essential content of the error messages will be the same.

Malformed XML

Symptom: The compiler reports an error message of the following form:

<filename>(<line>,<position>)
: error NS9037: An error occurred when parsing the XML file.

<error description text>

Cause: The ICF passed to the compiler, or one of the ADFs referenced by it, contains malformed XML. The error message usually indicates the name of the file containing the error, the line and position on which the error occurs, and some descriptive text indicating the nature of the problem.

Action: Correct the XML errors indicated in the error message and run the create or update command again. Opening the offending file in an XML editor (such as Management Studio) can sometimes be helpful if the nature of the error is not obvious from the message. XML editors typically offer features such as syntax highlighting and well-formedness checking that can help you find XML errors.

XML Schema Compliance Errors

Symptom: The compiler reports an error message of the following form:

<filename>(<line>,<position>)
: error NS9047: The XML does not match the XSD schema.
ProblemDescription: <error description text>

Cause: ICFs and ADFs must conform to XSD schemas provided by SQL-NS. This error occurs if one of the XML documents processed by the compiler does not conform to the corresponding XSD schema. The document may contain well-formed XML but may violate restrictions and constraints defined in the schema. For example, the document may contain elements not defined in the schema, or multiple instances of an element defined as a singleton in the schema.

Note

The XSD schemas for the ICF and ADF can be found in the XML Schemas directory of your SQL-NS installation. If you installed SQL Server in the default location, the full path to this directory is C:\Program Files\Microsoft SQL Server\90\NotificationServices\9.0.242\XML Schemas.

In the XML Schemas directory, the ICF schema is defined in the file ConfigurationFileSchema.xsd, and the ADF schema is defined in the file ApplicationDefinitionFileSchema.xsd.

Action: Correct the XML errors indicated in the error message and run the create or update command again. XML editors (such as Management Studio) can sometimes help identify schema compliance errors.

Missing Parameter Values

Symptom: The compiler reports an error message of one of the following forms:

[View full width]

<filename>(<line>,<position>): error NS9008: A required element in the     instance
   configuration is missing.
MissingParameter: <parameter name>

or

[View full width]

<filename>(<line>,<position>): error NS9023: A required element in the     application
   definition is missing.
MissingParameter: <parameter name>

Cause: You did not supply a value for a parameter used in the ICF or ADF.

Action: For every parameter in the ICF, specify a value in one of the following ways:

  • Pass a value for the parameter on the command line if you're using nscontrol create or nscontrol update.

  • Specify a value for the parameter using the parameters grid in the New Notification Services Instance dialog box or the Update Instance dialog box if you're using Management Studio.

  • Supply a default value for the parameter in the <ParameterDefaults> element of the ICF.

For every parameter used in the ADF, specify a value in one of the following ways:

  • Declare the parameter and supply a value in the <Parameters> element within the application's <Application> declaration in the ICF.

  • Supply a default value for the parameter in the <ParameterDefaults> element of the ADF.

For more information on parameters, see the "Parameters" section (p. 104) in Chapter 4, "Working with SQL-NS Instances."

SQL Errors During Compilation

Compiling an instance for the first time involves creating the instance and application database objects from the information in the ICF and ADFs. Updating an instance involves altering the database objects themselves or the information in them. It's possible that the compiler may experience problems connecting to the SQL Server or creating the required database objects. This section covers SQL-related problems that you might encounter during the compilation of an instance.

Unable to Connect or Log In to the SQL Server

Symptom: The compiler reports an error message indicating problems connecting to the SQL Server. The error message has the following form:

Notification Services failed to open a connection to SQL Server.
Server Name: <sql server instance>
Failed to connect to server <sql server instance>.

<error description text>

Cause: The create or update command failed to connect to the SQL Server instance specified in the <SqlServerSystem> element of the ICF.

Action: The first step in diagnosing connectivity problems is to attempt to connect to the SQL Server instance using Management Studio. When doing so, make sure that you attempt the connection with the same credentials that the create or update command uses (either the credentials of the user running the command, if you're using Windows Authentication, or the development account's SQL username and password if you're using SQL Server Authentication).

If you are able to connect using Management Studio, you should do the following:

  • Verify that the SQL Server name is specified correctly in the <SqlServerSystem> element of the ICF (if you're using a named instance of SQL Server, be sure to include the instance name).

  • If you're using the command-line tools and SQL Server Authentication, verify that the correct SQL username and password are passed to the nscontrol create or nscontrol update command.

  • If you're using Management Studio and SQL Server Authentication, verify that the Object Explorer from which you're launching the SQL-NS commands is connected to the SQL Server with the correct username and password.

If you are not able to connect to the server from Management Studio, perform the following checks:

  • Verify that the SQL Server is running.

  • Verify that there is network connectivity between the machine running the SQL-NS tools and the SQL Server machine.

If you are able to reach the SQL server from Management Studio but are not able to log in, make sure that the account you're using to connect has been granted login access to the server. For information on granting login access and other permissions, see the "Granting Permissions" section (p. 92) in Chapter 4.

Permission Denied Errors During Compilation

Symptom: The compiler reports an error message indicating that permission to create a database object was denied. The error message will vary in form, depending on the object being created.

Cause: The account that the create or update command is using to connect to the SQL Server does not have sufficient permissions to create the necessary database objects.

Action: Make sure the account that the create or update command uses to connect to the database is a member of the sysadmin fixed server role. See the section "The Development Account" (p. 35) in Chapter 2, "Getting Set Up," for information on how to create a development account and grant sysadmin membership.


Troubleshooting Event Submission

This section deals with problems that occur while trying to submit events into a SQL-NS application.

Event Provider Is Not Running

This section covers the class of event submission problems that relate to event providers not running.

The SQL-NS Service Hosting the Event Provider Is Not Started

Symptom: Event providers are not running, and the SQL-NS service is either stopped or paused (you can check the service state using the SQL Server Configuration Manager).

Cause: Either the SQL-NS service was never started, or it was stopped or paused explicitly. When the service is installed at registration time, it is not configured to start automatically on boot. Unless you manually changed the service's startup type to Automatic, the service will need to be restarted if a reboot occurred since it was last running.

Action: Start the SQL-NS service, either through the SQL Server Configuration Manager or by invoking the net start NS$<InstanceName> command from a command prompt.

Tip

If you want the SQL-NS service to start automatically when the machine boots, you can configure it for automatic startup, using the Services applet in Administrative Tools. When configuring the SQL-NS service for automatic startup, you must configure the SQL Server service as a dependency if the SQL Server runs on the same machine. Otherwise, the service control manager may attempt to start the SQL-NS service before the SQL Server at boot time. If this happens, the SQL-NS service will fail to start because it will not be able to connect to the SQL Server to access the instance and application database objects.

Event Providers Are Not Enabled

Symptom: Event providers are not running, and their status is reported as Disabled or Disable Pending.

Cause: Either the event providers were never enabled after the instance was created or updated, or they were explicitly disabled.

Action: Use the nscontrol enable command to enable the event providers from the command line. Alternatively, use the Instance Properties dialog box to enable the event providers in Management Studio. The section "Enabling and Disabling Components" (p. 492) in Chapter 14 provides details on the use of these tools.

Incorrect Schedule Supplied for Scheduled Event Provider

Symptom: A hosted, scheduled event provider does not run, or has not run for long period of time.

Cause: An invalid schedule may have been supplied in the event provider's declaration in the ADF.

Action: Examine the event provider's declaration in the <HostedProvider> element in the ADF. Verify that the correct start time and interval are specified.

A Built-In Event Provider Is Unable to Start

Symptom: The Application Event Log on the event provider's host machine contains entries from the NotificationServices event source with event IDs 2001, 2991, 2992, 2981, 2982, or 2983, or any other entry indicating a problem starting the FileSystemWatcher, SQLProvider, or AnalysisServicesProvider.

Cause: Either incorrect arguments were specified for the built-in event provider in its ADF declaration, or the provider encountered an error during its startup operations.

Action: Examine the event log messages for details about the nature of the errors. You may have to edit your ADF so as to supply the correct arguments to the event provider and then update the instance.

A Custom Event Provider Is Unable to Start

Symptom: The Application Event Log on the event provider machine contains entries from the NotificationServices event source with event IDs 2005, 2006, 2007, 2008, 2009, 2010, 2012, 2015, 2017, 2023, or 2024, or any other entry indicating an error loading, instantiating, or initializing a custom event provider.

Cause: SQL-NS was unable to load the event provider class or encountered an error while trying to initialize the provider.

Action: Examine the event log messages for details about the nature of the errors. If the error messages indicate that SQL-NS was not able to instantiate the event provider class, verify that the <ClassName> and <AssemblyName> elements in the event provider's declaration in the ADF specify the correct values. Remember that the class name must include any namespace prefix, and the assembly name must specify a full path or a strong name.

If the event provider was compiled with a later version of the .NET Framework than the one SQL-NS uses (version 2.0.50727 at the time of this writing), you might see an error message saying that the specified event provider class could not be instantiated or was not found in the given assembly. If this is the case, you need to edit the NSService.exe.config file to configure the service to load components built with your version of the .NET Framework. The NSService.exe.config file is located in the binaries subdirectory of your SQL-NS install directory. Open the file in a text or XML editor and add a <startup> element as follows:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <startup>
        <supportedRuntime version="v2.1.20321" />
        <supportedRuntime version="v2.0.50727" />
    </startup>
    <system.diagnostics>
        <switches>
            <add name="LogAdministrative" value="2" />
            ...
        </switches>
    </system.diagnostics>
</configuration>

Make sure there is a <supportedRuntime> element listed in the <startup> element that lists the version of the .NET Framework you are using. After editing the NSService.exe.config file, you must restart the service for the changes to take effect.

Note

The version number shown in the first <supportedRuntime> element in the preceding snippet of the NSService.exe.config file (2.1.20321) is just an example. It does not necessarily correspond to an actual or planned future version of the .NET Framework.

If SQL-NS was able to locate and load the provider but an error occurred during its initialization, you may have to debug your event provider class. Refer to the "Debugging a Custom Component" topic in the SQL-NS Books Online for instructions on how to do this.

Event Providers Do Not Submit Events

The problems covered in this section have to do with event providers that run but do not submit events as expected.

SQL Errors During Event Submission

Symptom: The Application Event Log on the event provider's host machine contains entries from the NotificationServices event source with event IDs in the 2000 range and SQL error messages in their descriptions.

Cause: The event provider was unable to perform a database operation required to submit events. The following are possible causes:

  • The event provider was unable to connect to the SQL Server.

  • The event provider encountered a permissions problem while executing SQL operations.

  • A database operation timed out.

  • A database operation generated a runtime error.

Action: Examine the event log messages for details about the nature of the errors.

If the event log messages indicate that the event provider was not able to connect to the SQL Server, follow the steps for diagnosing connectivity problems outlined in the "Unable to Connect or Log In to the SQL Server" section (p. 508) earlier in this chapter. When doing so, note that hosted event providers attempt to connect to the SQL Server using the credentials of the user under which the SQL-NS engine is running, if Windows Authentication is used. If SQL Server Authentication is used, hosted event providers connect using the SQL username and password supplied at registration time.

If the event log messages indicate permissions problems, see the "Event Provider Security" section (p. 236) in Chapter 8, "Event Providers," for information on the required permissions and database roles for event providers. Make sure you grant the correct permissions to the account the event provider uses to connect to the SQL Server.

If the event log messages indicate a SQL timeout on one or more commands, your SQL Server might be under too much load to execute the commands in a timely manner. You need to investigate the cause of the load on your SQL Server and take measures to alleviate it.

If the event log messages indicate other SQL runtime errors, the body of the error messages usually give clues as to the nature of the problems. The solutions might involve changes to the configuration of the event provider or to the event provider itself (in the case of custom event providers).

Other Event Provider Runtime Errors

Symptom: The Application Event Log on the event provider's host machine contains entries from the NotificationServices event source with event IDs in the 2000 range, containing information about runtime errors that the event provider encountered.

Cause: One or more of the event provider's runtime operations failed.

Action: The event log messages usually give a detailed description of the errors. The problem might be with the event provider itself, or with the event source from which it reads.

No Events Available from External Event Source

Symptom: The event provider is running but not submitting events. There are no errors in the Application Event Log.

Cause: The external event source from which the provider reads events is not supplying any new event data. For example, a stock event provider that reads stock data from a stock market feed might not be receiving events from that feed.

Action: The solution to this problem depends on the nature of the external event source and how events are read from it. The problem might be that the provider cannot connect to the event source, that the event source is not running, or that there really are no new events to submit.


Troubleshooting Notification Generation

This section covers problems that might occur while generating notifications. Some of these are problems with the generator itself, and others are problems with the input it is given and the logic that it executes.

The Generator Is Not Running

This section covers problems that prevent the generator from running.

The SQL-NS Service Hosting the Generator Is Not Started

Refer to the earlier section "The SQL-NS Service Hosting the Event Provider Is Not Started" (p. 510) and apply the information given there to the SQL-NS Service hosting the generator.

The Generator Is Not Enabled

Symptom: The generator is not running, and its status is reported as Disabled or Disable Pending.

Cause: Either the generator was never enabled after the instance was created or updated, or it was explicitly disabled.

Action: Use the nscontrol enable command to enable the generator from the command line. If you want to enable only the generator, without enabling other components, you can use the -generator option. Alternatively, use the Instance Properties dialog box to enable the generator from Management Studio. The section "Enabling and Disabling Components" (p. 492) in Chapter 14 provides details on the use of these tools.

No Notifications Are Generated

This section covers situations in which the generator is running, but no notifications are actually generated.

There Are No Events to Process

Symptom: No notifications are generated, and there are no events in the event tables or event chronicles.

Cause: Either no events were submitted into the system, or the event chronicle rules did not update the event chronicles correctly.

Action: If there are no events in the events tables, refer to the earlier "Troubleshooting Event Submission" section (p. 510) for an explanation of some possible causes.

If there are events in the events table, but the event chronicles do not contain the expected data, some of the event chronicle rules may have encountered runtime errors when they were invoked. The common rule firing errors are covered in the sections "A Rule Timed Out" (p. 516) and "An Error Occurred During Rule Execution" (p. 516), later in this chapter. Consult those sections and apply their recommendations to diagnose problems executing the chronicle rules.

Even if the chronicle rules execute without errors, they may still contain logic flaws that cause them to produce incorrect results. Examine the rules' statements, and use the SQL-NS debugging stored procedures to execute them manually and check their results. Chapter 11, "Debugging Notification Generation," provides information on how to use the debugging stored procedures.

There Are No Subscriptions to Process

Symptom: No notifications are generated, and there are no errors in the Application Event Log.

Cause: There might not be any subscriptions in the system, or the subscriptions that exist might be disabled or not be scheduled to run at the current time.

Action: Examine the subscription tables to determine whether there are subscriptions in the system. If not, add subscriptions using the Subscription Management Interface, a T-SQL script that inserts into the subscription views, or the SubscriptionLoader tool supplied with this book's source code. For information on the SubscriptionLoader tool, see the section "Testing Scheduled Subscriptions" (p. 176) in Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State."

If the subscriptions table does contain subscriptions but they are disabled (the values in the Enabled column are all 0), they will not be matched to produce notifications. Enable the subscriptions via the SQL-NS Subscription Management API (covered fully in Chapter 7, "The SQL-NS Subscription Management API") or by updating their Enabled values to 1 using T-SQL statements against the subscription views.

If the subscriptions in the system are all scheduled subscriptions, their schedules might be such that they are not due to be evaluated at the current time. You can use the NSScheduledSubscriptionList and NSScheduledSubscriptionDetails stored procedures to determine which subscriptions are scheduled to run at the current time (refer to the "Using the Scheduled Subscription Debugging Stored Procedures" section, p. 411, in Chapter 11 for more information).

Subscription and Event Data Do Not Match

Symptom: There are events in the system, the event chronicles are updated as expected, and the subscriptions tables contain valid subscriptions to process, but no notifications are generated, and no errors appear in the Application Event Log.

Cause: The events and subscriptions are matched based on the logic defined in the match rules. The match rule might contain a logic flaw, or it might simply be the case that the event data being submitted does not match any of the defined subscriptions.

Action: Examine the event and subscription data and verify the results of the match rules using the debugging stored procedures (described in Chapter 11). If the rule logic contains an error that prevents event and subscription data from being matched as they should, you will have to edit the rules and update the application. If the rule logic is correct, you should add different subscriptions or change the incoming event data to force some matches.

A Rule Timed Out

Symptom: The Application Event Log on the generator machine contains an entry with event ID 4045 and an error message indicating a rule timeout.

Cause: The execution of a match rule or chronicle rule took longer than the allowed execution time.

Action: Examine the event message to determine which of the rules timed out (the message will contain the rule text). By default, rules are allowed to execute for up to 30 minutes. If your application runs with very large volumes of data, or the rules perform very complex operations, they may well require longer than 30 minutes to run. In these cases, you can extend the timeout using the <ActionTimeout> element in the rule declaration. Refer to the SQL-NS Books Online for more information on the <ActionTimeout> element.

Caution

Carefully examine whether the rule execution time is reasonable before adjusting the timeout. Simply extending the rule timeout on a long-running rule might mask an underlying performance problem that causes the rule to take more execution time than it should.

The logic of a long-running rule might be expressed in an inefficient way, or the tables against which it operates may not be indexed properly. Also, the database system on which it runs may not be optimally configured. Refer to the "Indexes and Query Optimization" section (p. 425) in Chapter 12, "Performance Tuning," for information on rule optimization. The "Configuring a Database System for Deployment" section (p. 456) in Chapter 13, "Deploying a SQL-NS Instance," provides information on configuring a database system for best performance.

An Error Occurred During Rule Execution

Symptom: The Application Event Log on the generator machine contains an entry with event ID 4045 and an error message indicating that a runtime error occurred while executing the rule.

Cause: A runtime error occurred either in the generator framework that executes rules or in the rule itself.

Action: Examine the event message to determine which of the rules were involved in the error (the message will contain the rule text). The event description usually gives specifics about the type of error that occurred. When diagnosing the causes of rule errors, it can sometimes help to use the debugging stored procedures to run the rule manually and examine the application state before and after the rule execution. Chapter 11 provides information on how to use the debugging stored procedures. After you identify the problem with the rule, you will need to fix it in the ADF and update the instance.

The Event, Subscription, or Notification Throttle Was Exceeded

Symptom: Notifications are not generated, and the Application Event Log contains event messages indicating that one or more of the throttles were exceeded. Such event messages usually have event IDs 4120, 4121, or 64024.

Cause: One of the application-defined throttles was exceeded during the execution of the rule.

Action: Examine the event message to determine which throttle (event, subscription, or notification) was exceeded. The throttle values are declared in the ADF, in the <ApplicationExecutionSettings> element. If not declared, the throttle values default to 1000. You should either increase the throttle values if they are too low or examine your data and rule logic for problems if the number of events, subscriptions, or notifications being processed exceeds expected volumes. Throttles being exceeded might also be an indication of a security violation, so you should examine your system for evidence of suspicious activity. For more information about throttles, refer to the "Using Throttles" section (p. 474) in Chapter 13.


Troubleshooting Notification Distribution

This section covers problems in the distribution part of the SQL-NS pipeline. Problems in this area usually manifest themselves as notifications not being delivered, or not being delivered correctly.

The Distributor or the Components It Hosts Are Not Running

This section covers problems that prevent the distributor, or the content formatters and delivery protocols it hosts, from running.

The SQL-NS Service Hosting the Distributor Is Not Started

Refer to the earlier section "The SQL-NS Service Hosting the Event Provider Is Not Started" (p. 514) and apply the information given there to the SQL-NS Service hosting the distributor.

The Distributor Is Not Enabled

Symptom: The distributor is not running, and its status is reported as Disabled or Disable Pending.

Cause: Either the distributor was never enabled after the instance was created or updated, or it was explicitly disabled.

Action: Use the nscontrol enable command to enable the distributor from the command line. If you want to enable only the distributor without enabling other components, you can use the -distributor option. Alternatively, use the Instance Properties dialog box to enable the distributor from Management Studio. The section "Enabling and Disabling Components" (p. 492) in Chapter 14 provides details on the use of these tools.

The Distributor Is Unable to Start

Symptom: The distributor is not running, and the Application Event Log on the distributor machine contains entries in the 6000 range.

Cause: The distributor encountered errors during its startup processing.

Action: Errors with event IDs in the 6000 range indicate errors encountered by the distributor. Examine the descriptions in the event log messages to determine the causes of the errors.

A Custom Content Formatter or Delivery Protocol Is Unable to Start

Refer to the section, "A Custom Event Provider Is Unable to Start" (p. 511). That section describes various startup problems with custom event providers, but its recommendations also apply to custom content formatters and delivery protocols.

The Distributor Does Not Process New Notification Batches

If the generator produces new notification batches, but those batches do not get processed by the distributor, one of the problems described in this section might be the cause.

Notification Batch Generation Does Not Complete Successfully

Symptom: The notifications table contains notifications, but they are never processed by the distributor, and their delivery status codes remain at 0 (meaning that delivery has never been attempted). The notification batches table shows that the batches for these notifications have a status code of 5 (indicating that generation failed).

Cause: The generator began generating notifications for the notification batch but was unable to complete the batch successfully. Although there are notifications in the notifications table, the notification batch reflects an unsuccessful generation status, so the notifications are ignored by the distributor.

Action: Examine the Application Event Log for error messages indicating generation problems. Refer to the "Troubleshooting Notification Generation" section (p. 514) for possible causes of generator failures.

A Distributor Work Item Remains in an Incomplete State

Symptom: The distributor appears to be stuck processing one incomplete work item and does not attempt to process new notification batches. The row in the work items table (NSDistributorWorkItems, located in the application schema) for the work item being processed shows a status code of 1 (indicating it is being processed), a valid start time, and an end time value of NULL.

Note

All work items will be in this state for a short time while they are being processed. However, a work item remaining in this state for an extended period of time usually indicates a problem.

Cause: While processing one work item, the distributor encountered a situation from which its processing cannot advance. It might be in an infinite loop or waiting for an event or signal (which may never arrive). Because the distributor is stuck in this state, its work scheduling algorithm never runs, so new notification batches are not seen.

Action: The problem is most likely in the implementation of a custom delivery protocol or content formatter running in the distributor. The custom component probably has not returned control to the distributor from one of its methods. Stopping and restarting the SQL-NS engine will alleviate the condition and allow the distributor to proceed, but if a bug does exist in the delivery protocol or content formatter implementation, it is likely that the distributor will get into this state again. Examine the custom component code to determine the problem. Refer to the "Debugging a Custom Component" topic in the SQL-NS Books Online for information on how to debug custom content formatters and delivery protocols.

Notifications Are Not Delivered as Expected

This section describes a range of problem situations that cause the distributor to deliver notifications incorrectly. In these cases, the distributor might deliver the wrong number of notifications, send them to the wrong destinations, or send them at unexpected times.

The Subscriber Device Names Specified in the Notification Data Are Invalid

Symptom: Notification batches are generated successfully, the distributor creates work items from them, and these work items appear to be processed, but no notifications are actually delivered. Events with Event ID 4048 are written to the Application Event Log, indicating that some notifications cannot be delivered.

Cause: The notification data contains invalid subscriber device names. Specifically, the subscriber devices identified by the SubscriberId and DeviceName columns in the notifications table do not exist in the subscriber devices table (NSSubscriberDevices) in the instance schema. This can happen because the match rule inserted invalid device names into the notifications view, or the subscriber devices were deleted or renamed after the notifications were generated.

Because the distributor cannot locate the subscriber device records for the notifications, it is unable to determine which delivery channels should be used for their delivery and therefore cannot assign them to distributor work items (recall that notifications are assigned to work items on the basis of their target delivery channels). Thus, the distributor work items created from the notification batch all appear to be empty. They are marked as successfully processed but no notifications are delivered.

Action: Verify that the device names used in the match rules are correct and that the subscriber device records have not been deleted or renamed. The combination of the subscriber ID and device name in the notification data must match one of the rows in the NSSubscriberDevices table in the instance schema.

Subscriber Devices Are Configured for Incompatible Delivery Channels

Symptom: Notification batches are generated successfully, but some notifications are not included in any distributor work item and are never delivered.

Cause: Each notification is associated with a particular delivery channel based on a subscriber device. The notifications not delivered are targeted at delivery channels that use protocols the notification class does not support: The protocols used by those delivery channels do not appear in the list of supported protocols specified in the <Protocols> element of the notification class declaration in the ADF.

Action: Verify that the subscriber devices are configured to the correct delivery channels and that the delivery channel definitions in the ICF specify the correct protocols. If it is your intention that some protocols are not supported by the notification class, your SMI should prevent users from entering subscriptions associated with incompatible devices.

If the notification class should support all protocols, you have to change the notification class declaration to include the missing protocol support. Add a <Protocol> element to the notification class's <Protocols> element for each protocol and supply the appropriate protocol fields and configuration settings. See the "Notification Delivery" section (p. 149) in Chapter 5, "Designing and Prototyping an Application," for more information on declaring protocol support. After adding the required protocol configuration information, you must update the instance.

A Timeout Occurs While Reading Notification Data

Symptom: Notifications are not delivered, and the Application Event Log contains event messages from the NotificationServices event source with event ID 6074. The error message indicates that a SQL timeout occurred.

Cause: The SQL command that reads the set of notifications in a distributor work item is taking too long to execute.

Action: By default, the SQL command that reads the notifications in a distributor work item times out after a period of 15 minutes. Depending on the volume of data in your application, this command might reasonably require more time to execute. If this is the case, you can adjust the timeout by means of the <WorkItemTimeout> element in the ADF. Refer to the SQL-NS Books Online for more information on this element.

If you believe the default timeout should be adequate, or you've adjusted the timeout to a reasonable value and the command still takes too long, the following are possible explanations:

  • The notification batches are too large. On SQL-NS Enterprise Edition, you can control the notification batch size by means of the <NotificationBatchSize> element in the ADF. For more information on setting notification batch sizes, see the "Notification Batch Size" section (p. 439) in Chapter 12.

    Distributor logging is putting too much load on the SQL Server. You can reduce this load by adjusting the options in the <DistributorLogging> element in the ADF. For more information on controlling distributor logging, see the "Distributor Logging Options" section (p. 445) in Chapter 12.

    Your database system is not configured optimally. See the "Configuring a Database System for Deployment" section (p. 456) in Chapter 13 for information on configuring a database system for best performance.

Timeouts Occur While Updating Notification Delivery Status

Symptom: The delivery status of some notifications are not updated, and the Application Event Log contains event messages from the NotificationServices event source with event ID 6076. The event descriptions indicate that SQL timeouts occurred.

Cause: A timeout occurred on the SQL command that updates the notification delivery status.

Action: The SQL command that updates the notification delivery status used a fixed timeout of 5 minutes. This cannot be adjusted.

If this command times out, the following are possible explanations:

  • Your custom delivery protocol is reporting delivery status for too many notifications at once. Change the delivery protocol implementation to pass fewer NotificationStatus objects in each invocation of the distributor's notification status callback.

  • Distributor logging is putting too much load on the SQL Server. You can reduce this load by adjusting the options in the <DistributorLogging> element in the ADF. For more information on controlling distributor logging, see the "Distributor Logging Options" section (p. 445) in Chapter 12.

  • Your database system is not configured optimally. See the "Configuring a Database System for Deployment" section (p. 456) in Chapter 13 for information on configuring a database system for best performance.

The Distributor Delivers Duplicate Notifications

This section deals with problems in which subscribers receive duplicate notifications from a SQL-NS application.

Duplicate Notifications Are Being Generated

Symptom: The notifications table shows the same notification data generated multiple times for the same subscriber.

Cause: There are several possible reasons for duplicate notifications being generated:

  • The event provider submitted the same events multiple times.

  • The event chronicles are not maintained correctly, causing old data to generate repeated matches.

  • Duplicate subscriptions exist (either because a subscriber entered the same subscription more than once or because a bug in the SMI causes duplicate subscription records to be created).

  • The match rule contains a logic error that results in multiple rows being inserted into the notifications view for a single match.

Action: Examine the events tables to determine whether duplicate event data is being submitted. This might indicate an event provider problem, or the event provider may have obtained duplicate event data from the event source. If you control the event source, you can ensure that it does not communicate duplicate events to the event provider. Otherwise, you can filter out duplicate events in the event provider code (in the case of a custom event provider) or in the event chronicle rules.

If the problem is not duplicate events, examine your event chronicle tables and rules to make sure that they are working as you expect. If old data is not deleted from the chronicles, or filtered out by the match rule logic, it can cause the same notification to be generated more than once.

If the events and event chronicles appear correct, examine the subscriptions. Duplicate subscriptions will, in general, cause duplicate notifications to be generated. Duplicate subscriptions may have been entered by users (accidentally or intentionally) or might result from bugs in the SMI. If you want to prevent users from entering duplicate subscriptions, you can add validation logic to your SMI that checks for and eliminates duplicates.

Finally, if the events, event chronicles, and subscriptions look correct, the problem might be in the match rules. If a match rule inserts more than one row into the notifications view for each match, duplicate notifications can result. Examine the match rule logic and use the debugging stored procedures (described in Chapter 11) to find such problems.

The Same Notifications Are Being Delivered More Than Once

Symptom: The notifications table does not contain unexpected duplicates, but the same notifications are delivered more than once. It is likely that the notification distribution views will indicate multiple delivery attempts for each notification. (For information about using the distribution views, see the section "The Notification Distribution Views," p. 360, in Chapter 10, "Delivery Protocols.")

Cause: There are several possible explanations for a single notification (a single row of data in the notifications table) being delivered multiple times:

  • The notification was successfully delivered once, but its delivery status was not recorded. This made the notification appear as though it had not been sent, and a subsequent retry attempt delivered the notification again. Failure to record a notification's delivery status can result from an error or timeout while executing one of the distributor's SQL commands, or from a custom delivery protocol failing to report delivery status via the notification status callback.

  • A custom delivery protocol incorrectly performed multiple delivery operations when it handled the notification data.

  • The external delivery system, to which the delivery protocol sent the notification, delivered it to its final recipient more than once.

Action: If the Application Event Log contains event messages from the NotificationServices event source with event ID 6076, the distributor encountered errors while recording the delivery status of some notifications. Retry attempts may have delivered those notifications again, even if they were successfully delivered on the first attempt. Examine the descriptions in the event messages to determine the causes of the errors. If the failures to update delivery status were due to SQL timeouts, refer to the "Timeouts Occur While Updating Notification Delivery Status" section (p. 521) for information on how to correct the problem.

If the Application Event Log contains no errors related to recording delivery status and your application uses a custom delivery protocol, check the delivery protocol's implementation to make sure it reports notification status for all notifications it handles. Refer to the section, "The Custom Delivery Protocol Interface," (p. 373) in Chapter 10 for information on how custom delivery protocols should report notification status.

Also, make sure that the custom delivery protocol performs only one delivery operation on each notification passed to it by the distributor. For information on how to debug your custom delivery protocol, see the "Debugging a Custom Component" topic in the SQL-NS Books Online.

If your application delivers notifications to an external delivery system (such as a web service that provides the routing and delivery to the final recipient), this delivery system might be responsible for sending the notification more than once. If this is the case, the distribution log in your SQL-NS application will only show one delivery attempt, but the final recipient may have received the notification more than once. There might be a problem with the delivery instructions your SQL-NS delivery protocol sends to the external delivery system, or the external delivery system may not be operating correctly.

Notifications Are Delivered at Unexpected Times

This section deals with problems that cause notifications to be delivered long after they should have, or at times that don't match the schedules users intended for their scheduled subscriptions.

Generator Fall-Behind is Causing Notifications to be Generated Late

Symptom: The generator quantum clock has fallen behind and as a result, the generator is producing notifications from event batches that arrived a long time in the past. The distributor is delivering the resulting notification batches as soon as they are ready.

Cause: Notifications are generated later than they should, because the generator's quantum clock has fallen behind. This can happen if quantum processing takes longer than the quantum duration, or if the generator restarts after a period of downtime.

Action: You can get the quantum clock back on schedule by specifying quantum limits in the ADF. These control how far behind the quantum clock is allowed to fall before the generator skips quantums. For information on using quantum limits, see the "Quantum Limits" section (p. 437) in Chapter 12.

In addition to getting the generator back on schedule, it's even more important to understand why the generator fell behind in the first place, if downtime was not the cause. Most likely, the application's rules are performing poorly, causing quantum processing to take longer than it should. The rules might be expressed in an inefficient way, or the tables against which they operate may not be indexed properly. Also, the database system may not be optimally configured. Refer to the "Indexes and Query Optimization" section (p. 425) in Chapter 12 for information on rule optimization. The "Configuring a Database System for Deployment" section (p. 456) in Chapter 13 provides information on configuring a database system for best performance.

The Distributor Is Delivering Old Notifications After Downtime

Symptom: After a prolonged period of downtime, the distributor starts delivering notifications that were generated much earlier.

Cause: While the distributor component was not running, the generator was still generating notifications. When the distributor was restarted, it began processing this backlog of old notifications and delivering them late.

Action: If the semantics of your application dictate that notifications older than a certain age should not be delivered, you should specify a notification expiration age in the <ExpirationAge> element in the ADF. See the "Delivery Failure: Retry and Notification Expiration" section (p. 359) in Chapter 10 for more information about setting notification expiration ages. If you do not specify an expiration age, the notifications never expire, and the distributor attempts to send them no matter how old they are.

A Scheduled Subscription Specifies the Wrong Time Zone

Symptom: Notifications are being generated at the wrong time for a scheduled subscription.

Cause: The subscription definition specifies the wrong time zone. Either the user entered the subscription incorrectly, or the SMI has a bug that caused the wrong time zone information to be recorded.

Action: Examine your subscription data to make sure that the time zones on the subscription schedules are correct. You can use the NSScheduledSubscriptionList and NSScheduledSubscriptionDetails stored procedures to determine which subscriptions are scheduled to be evaluated at a given time (refer to the "Using the Scheduled Subscription Debugging Stored Procedures" section, p. 411, in Chapter 11 for more information on these stored procedures). You might have to debug your subscription management application to ensure that it records the correct time zone information based on users' input.

Notifications Are Delivered via the Wrong Delivery Channel

This section deals with problems that cause the distributor to use the wrong delivery channel when delivering notifications.

Subscriber Devices Are Not Configured Correctly

Symptom: Notifications are generated as expected, but delivered via the wrong delivery channel.

Cause: Subscriber devices records, in the NSSubscriberDevices table in the instance schema, specify the wrong delivery channel names. Either invalid user input or a bug in the SMI caused the subscriber devices to be configured to the wrong delivery channels.

Action: Examine the NSSubscriberDevices table in the instance schema to see the delivery channel specified for each device. If this information is incorrect, you will have to update the subscriber device records. Verify that your SMI is recording subscriber device information correctly.

The Match Rule Inserts the Wrong Subscriber Device Name into the Notifications View

Symptom: Each individual subscriber device record is correctly configured, but notifications are being sent to the wrong delivery channels.

Cause: The match rule may be inserting the wrong subscriber device names into the notifications view. Check the data in the notifications table to see the values inserted.

Action: Examine your match rules to determine where the device name is coming from. A logic error in the rule might cause the wrong name to be inserted, or, if your rule obtains the device name from the subscriptions table, the wrong name may have been recorded in that table.

Getting Help From the SQL-NS Development Community

As SQL-NS adoption has grown, the community of people knowledgeable about the product has grown too. There are already several SQL-NS experts, both within Microsoft and at other companies, willing to help others solve SQL-NS problems. The best way to reach these people is through the SQL-NS public newsgroup and the MSDN discussion forum.

The SQL-NS public newsgroup name is microsoft.public.sqlserver.notificationsvcs. You can access the newsgroup using any news reader. Alternatively, you can use the web-based MSDN discussion forum dedicated to SQL-NS topics, located at http://forums.microsoft.com/msdn/ShowForum.aspx?ForumID=97.

Generally, any question about SQL-NS is fair game on the newsgroup or discussion forum. Typical posts range from questions about applicability of SQL-NS to a particular scenario, to specific coding or design questions. To make it easy for others to answer your questions, always be as specific as possible and include code snippets where applicable.

Remember that participation in the newsgroup can be a two-way endeavor. You can use the newsgroup to get information from others, but you can also benefit the whole SQL-NS development community by sharing the knowledge and experience you have. Don't worry if you don't think you're an expertif you know the answer to someone's question, don't hesitate to post it. The value of the newsgroup and discussion forum lie in their interactive nature and the open information exchange they facilitate. As you learn about SQL-NS and use it in your applications, I encourage you to participate in the community as much as you can.


Summary

This chapter describes a number of problems you might encounter when using SQL-NS. For each problem described, the underlying causes and recommended solutions are given. Of course, you may well run into problems that are not covered here. In those cases, you might find descriptions of related problems and solutions in this chapter that could help you.

Even if you can't find anything directly relevant to the particular problem you're experiencing in this chapter, it is my hope that the general approach to problem diagnosis and debugging described here (and throughout this book) will aid you in finding your own solutions. SQL-NS troubleshooting can be daunting at first but gets easier as you learn more about the way the system works. Every problem you encounter can be a learning experience. I encourage you to always look beyond the surface of the problems you run into and use every one of them to build on your SQL-NS knowledge.


Part IV: Advanced Notification Services Concepts

IN THIS PART


Chapter 16. Using Notification Services Management Objects

IN THIS CHAPTER

  • Overview of the NMO API

  • An NMO Sample Application

  • Using NMO to Define and Create an Instance

  • Using NMO to Administer an Instance

  • Testing the StockBrokerNMO Sample Application

  • Using NMO to Reflect the Contents of an Instance

This chapter introduces Notification Services Management Objectsthe new API in SQL Server 2005 that provides programmatic access to SQL-NS management operations. Using this API, you can embed the functionality of the SQL-NS tools (nscontrol and Management Studio) in your own applications.

The Notification Services Management Objects (NMO) API wraps the functionality of the SQL-NS create, register, enable, disable, update, export, and delete operations in a convenient, reusable class library. In addition, the API provides a way to define new SQL-NS instances programmatically (instead of using XML) and enumerate the properties of existing instances. In this chapter, we explore all these uses of Notification Services Management Objects.


Overview of the NMO API

The NMO API provides programmatic access to the following SQL-NS functions:

  • Defining new SQL-NS instances

  • Creating, registering, enabling, disabling, updating, exporting, and deleting SQL-NS instances

  • Enumerating existing SQL-NS instances and their subcomponents

This section provides an overview of the NMO API's class structure. The use of NMO for each of the specific functions in the preceding list is detailed in the subsequent major sections of this chapter. Defining and creating instances with NMO is covered in the section "Using NMO to Define and Create an Instance" (p. 537). The section "Using NMO to Administer an Instance" (p. 549) covers several of the other operations listed in the second bullet item: registering, enabling, disabling, updating, and deleting SQL-NS instances. Finally, the enumeration capabilities of the API are explained in the section "Using NMO to Reflect the Contents of an Instance" (p. 555).

Note

In learning NMO, you'll encounter very few new SQL-NS concepts. Rather, you'll learn new, programmatic mechanisms for doing familiar tasks.

NMO is part of a larger API framework called SQL Server Management Objects (SMO). SMO provides programmatic management capabilities for the fundamental SQL Server entities, such as databases, indexes, security roles, and file groups. In the case of databases, for example, classes in the SMO API encapsulate such management operations as backup and restore. NMO is the subset of SMO that provides management capabilities specific to SQL-NS instances. For documentation on SMO in general, consult the SQL Server Books Online.

The NMO API is distinct from the other parts of the SQL-NS API you've already seen (the classes and interfaces used to build subscription management interfaces, event providers, content formatters, and delivery protocols). Classes in the NMO API are defined in a different namespace than the other SQL-NS API classes, and their implementations are packaged in a separate assembly. The section "NMO API Assemblies and Namespaces" (p. 532) describes the assemblies and namespaces you'll reference during NMO development.

Classes in the NMO API

The classes in the NMO API represent the entities that make up a SQL-NS instance. For example, the API contains classes that encapsulate delivery channels, protocols, applications, event classes, and so on. These classes are arranged in a containment hierarchy that reflects the structure of the elements in the ICF and ADF schemas. This hierarchy is shown in Figure 16.1. In the figure, collections are represented by three stacked boxes. Singletons appear as single boxes.

Figure 16.1. Classes in the NMO API.

[View full size image]

Note

In Figure 16.1, some class names are split across multiple lines to make them fit within their containing boxes. Also, for the sake of space, some class names are abbreviated. Specifically, the box representing the class HostedEventProviderArgument is labeled "HostedEventProv...Argument" and the box representing the class SubscriptionConditionScheduledRule is labeled "SubscriptionConditionSched...Rule".

The relationships depicted by the connecting lines in Figure 16.1 are containment relationships, not inheritance relationships. In other words, the line that connects the Instance class to the DeliveryChannel class indicates that DeliveryChannel objects are contained within Instance objects, not that DeliveryChannel inherits from Instance.

The containment relationships between NMO classes are also characterized as parent-child relationships. When one object is contained within another, we call the containing object the parent and the contained object the child. This can sometimes be confusing because the terms "parent" and "child" are usually used to denote inheritance relationships. However, in the context of NMO, they always refer to containment relationships.

Every NMO class has a Parent property that returns the parent object and various appropriately named properties that return child objects. As an example, consider a NotificationClass object that has a child ContentFormatter object. From the child ContentFormatter object, the parent NotificationClass object can be reached via the Parent property. From the parent NotificationClass object, the child ContentFormatter object can be reached via the ContentFormatter property.

When an object may have more than one child object of the same type, the children are grouped in collections. For example, an Application object may have several EventClass child objects. The Parent property on each of these child objects refers to the same Application object. On this parent Application object, the EventClass child objects are contained in a collection property named EventClasses.

The NMO class hierarchy is rooted at the Server class, which represents a SQL Server instance. Server is actually part of the broader SMO set of classes: it is not specific to NMO. The Server class has a NotificationServices property that contains a collection of NMO Instance objects.

The Instance class represents a SQL-NS instance and its child classes represent the entities in an instance, such as delivery channels, protocol definitions, and applications. The Application class represents a SQL-NS application, with child classes for the application-level entities, such as event classes, subscription classes, and distributors. Each NMO class has properties that encapsulate the various attributes of the entity it represents. For example, the EventField class, which represents fields in event class schemas, has properties for the field name, type, and type modifiers.

This chapter explores the programming patterns in which the NMO classes are commonly used. It is not a complete class reference. The code listings and text in this chapter cover many examples of NMO usage that illustrate the general functionality of the API. However, you won't see every class pictured in Figure 16.1 used or described in this chapter. Consult the SQL-NS Books Online for complete reference documentation on every NMO class, property, and method.

NMO API Assemblies and Namespaces

To use the NMO API, applications must reference the assemblies that contain the needed classes. The following two assemblies are usually required:

  • Microsoft.SqlServer.Smo.dll

  • Microsoft.SqlServer.ConnectionInfo.dll

The first assembly, Microsoft.SqlServer.Smo.dll, contains most of the classes in the SMO Framework, including the NMO classes (there isn't a separate NMO assembly). The Microsoft.SqlServer.ConnectionInfo.dll contains utility classes used to establish database connections. Classes from both assemblies are typically needed in programs that use the NMO API.

Most of the classes you'll encounter while working with NMO are defined in one of the following three namespaces:

  • Microsoft.SqlServer.Management.Nmo

  • Microsoft.SqlServer.Management.Smo

  • Microsoft.SqlServer.Management.Common

The first one is sometimes referred to as the NMO namespace. It contains all the NMO-specific classes. The second namespace contains many of the SMO classes, some of which are needed in NMO programming. The third namespace contains common utility classes, such as those used to establish database connections.

Note

An NMO-based application need not reference the main SQL-NS assembly (Microsoft.SqlServer.NotificationServices.dll) unless it uses classes in the main SQL-NS namespace, Microsoft.SqlServer.NotificationServices. (Many of the classes in the main SQL-NS namespace are covered in the earlier chapters of this book.)

In the previous SQL-NS release, some of the capabilities now provided by the new NMO API were provided as part of the core SQL-NS API. As a result, some overlap exists between the classes in the Microsoft.SqlServer.Management.Nmo and the Microsoft.SqlServer.NotificationServices namespaces. For example, both namespaces contain a class called DeliveryChannel.

In every case of overlap, the class in the Microsoft.SqlServer.NotificationServices namespace has been deprecated: the new NMO class should be used instead. Following the standard Microsoft procedure for API deprecation, the old classes that overlap with NMO are marked as deprecated in the SQL Server 2005 documentation. They will be removed from the documentation in the next release, and will be removed entirely in the release following that.

Whenever both the Microsoft.SqlServer.Management.Nmo and Microsoft.SqlServer.NotificationServices namespaces are in scope (for example, when both namespaces have been declared in using directives in a source file), you must use fully qualified names to disambiguate some class references. For example, you'll need to refer to the NMO DeliveryChannel class as Microsoft.SqlServer.Management.Nmo.DeliveryChannel. Using class names that exist in both namespaces, without fully qualifying them, will result in compile errors.


An NMO Sample Application

This chapter introduces a new sample application that illustrates the capabilities of the NMO API to define, create, register, enable, and delete a new SQL-NS instance. Think of this application as a single program that performs all the operations you normally do with separate SQL-NS commands to create an instance and prepare it to run.

The SQL-NS instance defined by the sample application is essentially the same StockBroker instance we looked at in Chapter 3, "The Simplest Notification Application: Stock Quotes." The same instance and application definitions are used, but here they are specified through NMO instead of XML. We also use the same steps we used in Chapter 3 to get the instance running. But instead of using the SQL-NS tools in Management Studio (or their command-line equivalents), we invoke these steps programmatically with the NMO API.

Although the SQL-NS instance we create in this chapter is the same in content as the one created in Chapter 3, we distinguish it with a unique name. Chapter 3's instance is called StockBroker; the instance we create in this chapter is called StockBrokerNMO.

The code for the StockBrokerNMO sample is packaged into a Visual Studio solution, located at C:\SQL-NS\Samples\StockBrokerNMO. The solution file is named StockBrokerNMO.sln. Open this in Visual Studio now and refer to it as you read through this chapter. This section explains the organization of the code in the solution and illustrates the basic structure of the application, before we explore the details of the NMO API.

The StockBrokerNMO solution has three main source files:

  • Program.csContains the main driver code for the application

  • Arguments.csContains classes used to parse the application's command-line arguments

  • StockBrokerNMO.csContains the StockBrokerNMO class that encapsulates the core functionality of the application

The outline of the StockBrokerNMO class is shown in Listing 16.1. Notice that the three namespaces associated with NMO and SMO are declared in using statements at the start of the code.

Listing 16.1. Structure of the StockBrokerNMO Class

...
using Microsoft.SqlServer.Management.Nmo;
using Microsoft.SqlServer.Management.Smo;
using Microsoft.SqlServer.Management.Common;

namespace StockBrokerNMO
{

   public class StockBrokerNMO

   {

      ...

      public StockBrokerNMO(
          string sqlServer,
          AuthenticationMode authenticationMode,
          string sqlUserName,
          string sqlPassword,
          string baseDirectoryPath,
          string serviceUserName,
          string servicePassword,
          AuthenticationMode serviceAuthenticationMode,
          string serviceSqlUserName,
          string serviceSqlPassword)
      {
          ...
      }

      public void Cleanup()
      {
          ...
      }

      public void Create()
      {
          ...
      }

      public void Register()
      {
          ...
      }

      public void Enable()
      {
          ...
      }
   }

}

The class consists of a public constructor and four public methods that implement the various operations required to create and administer the SQL-NS instance. We examine the implementations of the constructor and these methods (which use the NMO API) in the sections "Using NMO to Define and Create an Instance" (p. 537) and "Using NMO to Administer an Instance" (p. 549) later in this chapter.

The code in Program.cs drives the application by invoking the methods of the StockBrokerNMO class, as shown in Listing 16.2.

Listing 16.2. Calling the Methods of the StockBrokerNMO Class in the Program's Main() Driver Method

...
namespace StockBrokerNMO
{

    ...
    class Program
    {

        ...
        static int Main(string[] args))
        {

           ...
           try
           {

               // Instantiate the StockBrokerNMO class.
               StockBrokerNMO stockBrokerNMO = new StockBrokerNMO(
                   arguments["-sqlServer"].ModifierValue,
                   authenticationMode,
                   sqlUserName,
                   sqlPassword,
                   arguments["-baseDirectoryPath"].ModifierValue,
                   arguments["-serviceUserName"].ModifierValue,
                   arguments["-servicePassword"].ModifierValue,
                   serviceAuthenticationMode,
                   serviceSqlUserName,
                   serviceSqlPassword);

               // Do cleanup first, to get rid of any previous copy
               // of the instance.
               Console.WriteLine("Cleaning up Instance");
               stockBrokerNMO.Cleanup();

               // Create the instance.
               Console.WriteLine("Creating StockBrokerNMO Instance");
               stockBrokerNMO.Create();

             // Register the instance.
             Console.WriteLine("Registering Instance");
             stockBrokerNMO.Register();

             // Enable the instance.
             Console.WriteLine("Enabling Instance");
             stockBrokerNMO.Enable();

         }
         catch (Exception ex)
         {
             ...
         }

         ...
      }
   }
}

This code creates an instance of the StockBrokerNMO class, obtaining the constructor's parameter values from various command-line arguments. It then calls the Cleanup() method on the StockBrokerNMO class to clean up any existing copy of the StockBrokerNMO instance that might have been created by a previous run of the program. This clears the way to create a new instance, by calling the Create() method. Calls to the Register() and Enable() methods register and enable the instance, respectively.

For the purposes of this chapter, you can ignore the other code in Program.cs, not shown in Listing 16.2. Most of this code deals with processing and validating the application's command-line arguments.

The code in the StockBrokerNMO solution is compiled into a single executable, StockBrokerNMO.exe. Instructions for invoking StockBrokerNMO.exe and supplying the command-line arguments it requires are provided in the section "Testing the StockBrokerNMO Sample Application" (p. 554) after we've examined the usage of the NMO API in the rest of the code.


Using NMO to Define and Create an Instance

The classes in the NMO API can be used to define a new SQL-NS instance programmatically and then compile this definition into the corresponding SQL-NS database objects. Using NMO, you can embed the definition of an instance, as well as the functionality of the SQL-NS compiler, in your own applications.

This capability is useful if you need to construct an instance definition dynamically. For example, imagine a scenario in which some aspects of a SQL-NS instance's deployment configuration, such as the number of delivery channels, aren't known at development time. Constructing a static ICF that declares the delivery channels simply isn't possible.

With NMO, you can instead write a program that builds the instance definition dynamically, based on options specified at deployment time. Among these options could be the number of delivery channels (and their related configuration information). The program could use this information to construct a tailored instance definition with the correct number of delivery channel declarations. Without NMO, implementing the equivalent functionality would require writing custom XML processing code that pieces together an ICF from a static template. NMO provides a much simpler programming interface for dynamic instance definition.

Defining a SQL-NS instance with the NMO API involves building a hierarchy of NMO objects. Each object represents one entity in the instance, such as a delivery channel or an event class. You create each object of the appropriate NMO class, fill in its properties, and then link it to a parent object. This process begins with an object of the NMO Instance class, which represents a whole instance.

After the NMO object hierarchy that defines the instance is built, you call the Create() method on the Instance object to compile the instance. This invokes the same compilation operations carried out by the standard instance creation tools, nscontrol create and the New Notification Services Instance command in Management Studio.

This section looks at the code that defines and creates the sample StockBrokerNMO instance. The code uses NMO classes to define the same information that was specified in the ICF and ADF for the StockBroker sample in Chapter 3. The code examples in this section highlight the various programming patterns involved in creating a SQL-NS instance with NMO, but do not cover every class and property. The StockBrokerNMO instance is representative of a simple SQL-NS instance, but does not use all the SQL-NS features. NMO classes dealing with constructs not used in the StockBrokerNMO instance, such as chronicles and scheduled rules, are not covered here. Refer to the SQL-NS Books Online for a complete class reference to the NMO API.

Defining an Instance with NMO

The code that defines the StockBrokerNMO instance is located in the StockBrokerNMO class's Create() method. Before looking at this method, we must examine the StockBrokerNMO class constructor, which establishes the connection to the SQL Server, from which all other operations proceed. The constructor's code is shown in Listing 16.3.

Listing 16.3. The StockBrokerNMO Class Constructor

public class StockBrokerNMO
{

   ...
   private Server server;
   ...
   public StockBrokerNMO(
       string sqlServer,
       AuthenticationMode authenticationMode,
       string sqlUserName,

       string sqlPassword,
       string baseDirectoryPath,
       string serviceUserName,
       string servicePassword,
       AuthenticationMode serviceAuthenticationMode,
       string serviceSqlUserName,
       string serviceSqlPassword)

    {
       ...
       // Establish a server connection.
       ServerConnection serverConnection = new ServerConnection();
       serverConnection.ServerInstance = sqlServer;

       if (AuthenticationMode.SqlServerAuthentication ==
           authenticationMode)

       {

          serverConnection.LoginSecure = false;
          serverConnection.Login = sqlUserName;
          serverConnection.Password = sqlPassword;

       }

       server = new Server(serverConnection);

    }
    ...
 }

The constructor creates a ServerConnection object which represents a SQL Server connection. ServerConnection is a class in the SMO library; it provides various properties that define a connection to a SQL Server. Specifically, the ServerInstance property specifies the SQL Server instance name; the LoginSecure, Login, and Password properties control the authentication method used to connect. When LoginSecure is false, and the Login and Password properties specify a SQL Server username and password, the connection uses SQL Server Authentication. Otherwise, Windows Authentication is used. The code in Listing 16.3 sets these properties based on the values of the constructor's parameters.

After the ServerConnection object is created, the StockBrokerNMO constructor creates an instance of the Server class. Whereas the ServerConnection object represents a connection to the server, the Server object represents the server itself. The Server constructor takes the ServerConnection object, which it uses to identify and connect to a particular SQL Server. Thereafter, the Server object can be used to manipulate that server and the entities associated with it.

The Server object created in the StockBrokerNMO constructor is stored in a private class member variable so that it can be used in other methods. For example, the Create() method uses it, as shown in Listing 16.4.

Listing 16.4. The Create() Method of the StockBrokerNMO Class

public class StockBrokerNMO
{

   ...
   const string InstanceName ="StockBrokerNMO";

   private Server server;
   private Instance instance;
   ...
   public void Create()
   {
       // Create a new instance object.
       instance = new Instance(
           server.NotificationServices,
           InstanceName);

       // Add the instance to the server's collection.
       server.NotificationServices.Instances.Add(instance);

       // Specify the instance database and schema names.
       instance.DatabaseName ="StockBrokerNMO";
       instance.SchemaName ="NSInstance";

       // Define the delivery channel.
       DeliveryChannel dc = new DeliveryChannel(
           instance,
           "FileChannel");
       dc.ProtocolName ="File";
       instance.DeliveryChannels.Add(dc);

       // Add the delivery channel arguments.
       DeliveryChannelArgument dcArg =
           new DeliveryChannelArgument(dc, "FileName");
       dcArg.Value = Path.Combine(
           baseDirectoryPath,
           "FileNotifications.txt");
       dc.DeliveryChannelArguments.Add(dcArg);

       // Define the StockWatcher application and add it
       // to the instance.
       Application stockWatcherApplication =
           BuildStockWatcherApplication();
       instance.Applications.Add(stockWatcherApplication);

       // Create the instance.
       instance.Create();
   }
   ...
}

The code in the Create() method creates an NMO Instance object to represent the StockBrokerNMO SQL-NS instance. It fills in the properties of the Instance object, and then creates all the child objects needed to fully define the instance. Among these child objects is an Application object that represents the instance's StockWatcher application. This Application object is created and populated by a helper method, BuildStockWatcherApplication(), that we examine in the following section, "Defining an Application with NMO" (p. 543).

As shown in Figure 16.1, the parent of an NMO Instance object is always a NotificationServices object attached to a Server object. The NotificationServices object isn't something that needs to be created in application codeit's a singleton that always exists as a property of a Server object.

The first line of the Create() method shown in Listing 16.4 creates the new NMO Instance object. The Instance constructor takes two arguments: the instance's parent NotificationServices object and the instance name. Here, the parent is specified as the NotificationServices property of the Server object created earlier and stored in the private member variable, server. The instance name is obtained from the string constant, InstanceName, defined at the top of the class. The newly created Instance object is stored in the StockBrokerNMO class's private member variable, instance. This makes it accessible to other methods in the class, as you'll see in the subsequent sections of this chapter.

Caution

The SMO Server class supports old versions of SQL Server. For example, a Server object can represent an instance of SQL Server 2000 or SQL Server 7.0. Although the Server class and other parts of SMO support releases of SQL Server prior to 2005, NMO does not. Accessing the NotificationServices property on a Server object that represents an old SQL Server (SQL Server 2000 or earlier) results in an exception of type InvalidSmoOperationException.

After the Instance object is created, it is added to the parent object's Instances collection. This illustrates how a parent-child linkage is established in an NMO object hierarchy: The parent object is specified when the child object is constructed, and the child object is then explicitly added to the appropriate collection in the parent object. Both steps are necessary to complete the linkage. This pattern is used throughout the code that creates the NMO object hierarchy to define the instance. In cases where a parent object can have only one instance of a particular type of child object (for example, a notification class may have only one content formatter), the child object is set as a property on the parent object, rather than added to a collection.

Having created the Instance object and added it to the parent object's Instances collection, the code in Listing 16.4 goes on to specify the instance's properties. For example, the next two lines specify the instance database name and schema name. Then the various child objects are created. In this example, only two child objects are needed: a delivery channel and an application.

The DeliveryChannel object is created using a constructor that again takes two arguments: a parent object and a name. Here the delivery channel's parent object is the instance, and the name is given as "FileChannel". The only property we need to set on the DeliveryChannel object, the ProtocolName, is assigned the constant "File" (which identifies the built-in File delivery protocol). The DeliveryChannel object is then added to the DeliveryChannels collection in the parent object (the instance). This establishes the two-way parent-child relationship.

The code in Listing 16.4 then creates a delivery channel argument, following a similar pattern. A DeliveryChannelArgument object is created, initialized, and then added to the parent's DeliveryChannelArguments collection. In this case, the parent is the delivery channel itself.

As mentioned earlier, the Application object that represents the instance's StockWatcher application is constructed and initialized by a helper method, BuildStockWatcherApplication(), which we examine in the next section. In Listing 16.4, the Application object returned from this method is added to the parent Instance object's Applications collection. This completes the definition of the instance.

The Instance object's Create() method is then called to compile the instance definition and create the corresponding database objects. This method is described in the section "Creating the Instance" (p. 549), after the discussion of the Application object and its children in the next section.

Tip

To see how the instance definition specified here in code compares with the XML definitions used in Chapter 3, refer to the files C:\SQL-NS\Samples\StockBroker\InstanceConfiguration.xml and C:\SQL-NS\Samples\StockBroker\StockWatcher\ApplicationDefinition.xml. As you'll see by comparing with the contents of these files, the code used here specifies essentially the same information as the XML definitions.

Note

NMO provides classes and properties for defining other instance-level entities not used in this chapter's example. Notably, the ProtocolDefinition class encapsulates custom delivery protocol declarations, the InstanceDatabaseOptions class defines the physical storage for instance database objects, and the ArgumentKey property specifies an argument key used to encrypt and decrypt arguments in the instance. You'll find more information on these and other NMO classes in the SQL-NS Books Online.

Defining an Application with NMO

The StockWatcher application in the StockBrokerNMO instance is defined in the BuildStockWatcherApplication() method, as shown in Listing 16.5. The code in this listing uses NMO objects to specify the various parts of the application, including the event, subscription, and notification classes. The application definition produced by this code is essentially the same as that specified in the ADF we examined in Chapter 3.

Listing 16.5. The BuildStockWatcherApplication() Method of the StockBrokerNMO Class

public class StockBrokerNMO
{

   ...
   private Application BuildStockWatcherApplication()
   {
       // Create the application object.
       Application application = new Application(
           instance,
           "StockWatcher");
      // We don't add the application to the instance's
      // applications collection here - that's done outside
     // this method.

    // Set the database and schema names.
   application.DatabaseName = "StockBrokerNMO";
   application.SchemaName = "StockWatcher";

   // Set the base directory path.
   application.BaseDirectoryPath = Path.Combine(
       baseDirectoryPath,
       application.Name);

   // Add the event class.
   EventClass ec = new EventClass(
        application,
        "StockPriceChange");
   application.EventClasses.Add(ec);

   // Add the event fields to the event class.
   EventField ef = new EventField(ec, "StockSymbol");
   ef.Type = "NVARCHAR(10)";

    ef.TypeModifier = "NOT NULL";
    ec.EventFields.Add(ef);

    ef = new EventField(ec, "StockPrice");
    ef.Type = "SMALLMONEY";
    ef.TypeModifier = "NOT NULL";
    ec.EventFields.Add(ef);

    // Add the subscription class.
    SubscriptionClass sc = new SubscriptionClass(
        application,
        "StockPriceHitsTarget");
    application.SubscriptionClasses.Add(sc);

    // Add the subscription fields.
    SubscriptionField sf = new SubscriptionField(
        sc,
        "StockSymbol");
    sf.Type = "NVARCHAR(10)";
    sf.TypeModifier = "NOT NULL";
    sc.SubscriptionFields.Add(sf);

    sf = new SubscriptionField(sc, "StockPriceTarget");
    sf.Type = "SMALLMONEY";
    sf.TypeModifier = "NOT NULL";
    sc.SubscriptionFields.Add(sf);

    // Add the event rule.
    SubscriptionEventRule rule = new SubscriptionEventRule(
       sc, "MatchStockPricesWithTargets");
    rule.Action = @"
    INSERT INTO [StockWatcher].[StockAlert]
    SELECT subscriptions.SubscriberId,
           N'DefaultDevice',
           N'en-US',
           events.StockSymbol,
           events.StockPrice,
           subscriptions.StockPriceTarget
    FROM   [StockWatcher].[StockPriceChange] events
    JOIN   [StockWatcher].[StockPriceHitsTarget] subscriptions
        ON events.StockSymbol = subscriptions.StockSymbol
    WHERE events.StockPrice >= subscriptions.StockPriceTarget";
    rule.EventClassName = ec.Name;

    sc.SubscriptionEventRules.Add(rule);

    // Add the notification class.
    NotificationClass nc = new NotificationClass(
        application,
        "StockAlert");
    application.NotificationClasses.Add(nc);

    // Add the notification fields.
    NotificationField nf = new NotificationField(
        nc,
        "StockSymbol");
    nf.Type = "NVARCHAR(10)";
    nc.NotificationFields.Add(nf);

    nf = new NotificationField(nc, "StockPrice");
    nf.Type = "SMALLMONEY";
    nc.NotificationFields.Add(nf);

    nf = new NotificationField(nc, "StockPriceTarget");
    nf.Type = "SMALLMONEY";
    nc.NotificationFields.Add(nf);

    // Define the content formatter and its arguments.
    nc.ContentFormatter = new ContentFormatter(
         nc,
         "XsltFormatter");

    ContentFormatterArgument cfArg =
        new ContentFormatterArgument(
            nc.ContentFormatter,
            "XsltBaseDirectoryPath");
    cfArg.Value = Path.Combine(
        application.BaseDirectoryPath,
        "XslTransforms");
    nc.ContentFormatter.ContentFormatterArguments.Add(cfArg);

    cfArg = new ContentFormatterArgument(
        nc.ContentFormatter, "XsltFileName");
    cfArg.Value = "StockAlert.xslt";
    nc.ContentFormatter.ContentFormatterArguments.Add(cfArg);

    // Add the protocols for the notification class.
    NotificationClassProtocol protocol =

        new NotificationClassProtocol(nc, "File");
    nc.NotificationClassProtocols.Add(protocol);

    // Add the nonhosted event provider.
    NonHostedEventProvider provider =
       new NonHostedEventProvider(
       application,
       "TestEventProvider");
    application.NonHostedEventProviders.Add(provider);

    // Define the generator.
    application.Generator = new Generator(
        application,
        "Generator");
    application.Generator.SystemName = Environment.MachineName;

    Distributor distributor = new Distributor(
        application,
        "Distributor1");
    distributor.SystemName = Environment.MachineName;
    distributor.QuantumDuration = new TimeSpan(0, 0, 15);
    application.Distributors.Add(distributor);

    // Specify the application's execution settings.
    application.QuantumDuration = new TimeSpan(0, 0, 15);

    return application;

  }
  ...
}

The code begins by creating a new Application object, specifying the Instance object as the application's parent and "StockWatcher" as the application name. Note that the Application object is not added to the parent object's Applications collection here. That is done after BuildStockWatcherApplication() returns, as shown in Listing 16.4.

After creating the Application object, the code fills in the object's properties, including the database name, schema name, and base directory path. It then creates child objects to represent the application's event, subscription, and notification classes, as well as its event provider, generator, and distributor.

The application's event class, StockPriceChange, is represented by an instance of the NMO class, EventClass. As you might expect, the Application object is specified as the EventClass object's parent in the call to its constructor. The EventClass object is added to its parent's EventClasses collection after it is created. As in the examples of the previous section, this establishes the two-way parent-child relationship.

The fields of the event class schema are represented by EventField objects. Properties on the EventField objects specify the field names, types, and type modifiers. The event field objects are children of the EventClass object and are added to its EventFields collection.

The application's subscription class and the fields of its schema are defined in much the same way. A SubscriptionClass object is created, added to the parent application's SubscriptionClasses collection, and child SubscriptionField objects are created for each subscription class field.

The subscription class also contains an event rule. This is represented by a SubscriptionEventRule object. After creating the SubscriptionEventRule object, the code sets its Action and EventClassName properties before adding it to the parent subscription class's SubscriptionEventRules collection. The Action property of the rule object is set to the text of the match rule, almost exactly as it would have been specified in the ADF. Notice, however, that the XML escape sequence, >, is no longer needed in place of the > character, as it was in the ADF; > can be used directly, because the rule text is not part of an XML document.

The definition of the notification class and its schema follows essentially the same pattern as the event and subscription classes. But the notification class also specifies a content formatter and a protocol.

The content formatter is represented by an instance of the ContentFormatter class. In this example, the ContentFormatter object specifies the built-in XsltFormatter as its name. Because each notification class can have just a single content formatter, NotificationClass does not have a collection of ContentFormatter objects. Instead, NotificationClass has a single ContentFormatter property, to which the new ContentFormatter object is directly assigned.

Had the ContentFormatter object referred to a custom content formatter class (instead of the built-in XsltFormatter), its AssemblyName property would have been set. Because XsltFormatter is a built-in formatter class, no assembly name is required, so the code in Listing 16.5 doesn't set the ContentFormatter object's AssemblyName property at all.

Note

The ContentFormatter object does not have an explicit ClassName property, as you might expect. Instead, the content formatter class name (XsltFormatter, in this example) is specified as the ContentFormatter object's name. In general, NMO classes and their properties mirror the structure (and naming conventions) of their counterpart ADF elements, but this is one exception.

The two required arguments for the XsltFormatter are represented by two ContentFormatterArgument objects. These are created, initialized with the appropriate values, and then added to the parent ContentFormatter object's ContentFormatterArguments property.

The protocol configuration for the notification class is represented by a single NotificationClassProtocol object. The protocol name, File, is given as the object name and the NotificationClass object is the parent. Because the File protocol configuration in the notification class does not require any protocol fields, nothing else is required before the NotificationClassProtocol object is added to the parent's NotificationClassProtocols collection. Had protocol fields been required, they could have been specified with ProtocolField objects. Also, if the notification class supported more than one protocol, additional NotificationClassProtocol objects could have been created and added to the NotificationClass object's NotificationClassProtocols collection in the same way.

The StockWatcher application configures three running components: a standalone event provider, a generator, and a distributor. These are represented by NonHostedEventProvider, Generator, and Distributor objects, respectively. Like the ContentFormatter object in the NotificationClass object, the Generator child object is a singleton in the parent Application object. This reflects the restriction that each application has only one generator. As you can see from Listing 16.5, the Generator object is created and assigned to the Application object's Generator property, rather than added to a collection.

You might be surprised to see that the Generator and Distributor objects in Listing 16.5 are assigned names ("Generator" and "Distributor1", respectively). This is a deviation from the ADF structure, in which <Generator> and <Distributor> declarations do not specify names. The SMO architecture requires that all objects be named and because NMO is part of SMO, this requirement extends to the NMO objects as well. When assigning names for Generator and Distributor objects, you may choose any value that complies with the rules for valid SQL Server identifiers. (See the "Identifiers [SQL Server]" topic in SQL Server Books Online for more details.) If an application has more than one distributor, each Distributor object must be assigned a unique name.

The application execution settings that can be specified in the <ApplicationExecutionSettings> ADF element are specified via properties on the Application object in NMO. NMO does not define a distinct ApplicationExecutionSettings class. In the example in Listing 16.5, the only execution setting specified is the generator's quantum duration. This is set via the Application object's QuantumDuration property.

The quantum duration value is represented as a TimeSpan object. In general, NMO uses TimeSpan objects to represent values that are specified in the XSD duration syntax in the ADF. Looking back a few lines in Listing 16.5, you'll notice that the distributor's quantum duration is also specified with a TimeSpan object, in the Distributor object's QuantumDuration property.

By the end of the BuildStockWatcherApplication() method, the Application object and its children have been completely populated. This completes the application definition. The Application object is returned to the caller (the StockBrokerNMO.Create() method). There it is added to the parent Instance object's Applications collection, as shown in Listing 16.4.

Creating the Instance

An instance defined through NMO can be compiled by invoking the Create() method on the Instance object. In the StockBrokerNMO example, you saw the code that invoked the Instance object's Create() method toward the end of Listing 16.4.

The Instance.Create() method invokes the SQL-NS compiler to validate and then compile the instance definition represented by the Instance object and all its children. If the instance definition is valid, the compiler creates the instance and application database objects on the SQL Server to which the Instance object is connected (the Server object whose NotificationServices property is the Instance object's parent).

Note

As you saw in previous chapters when you invoked the SQL-NS compiler using the standard SQL-NS tools (nscontrol create and the New Notification Services Instance command in Management Studio), compiling a SQL-NS instance can take several minutes. The Instance.Create() method may well take a similarly long time to complete.

If the compiler encounters an error as it creates the instance, the Instance.Create() method throws an exception. The exception object thrown indicates that the creation process failed. The inner exception object (accessed via the exception's InnerException property) usually provides more details as to the cause of the failure.


Using NMO to Administer an Instance

Most capabilities of nscontrol and the SQL-NS commands in Management Studio are available through NMO. In fact, the administration operations exposed in NMO share a common implementation with these standard SQL-NS tools. Invoking an operation via NMO is equivalent to invoking the corresponding nscontrol or Management Studio command. This means you can "mix and match" the use of NMO with the standard tools: an instance created via NMO can be managed with the standard tools, and vice versa.

Only two SQL-NS administration operations are not exposed in NMO:

  • RepairThe operation that rebuilds instance metadata in the system tables (available only through nscontrol)

  • UpgradeConverting a compiled instance from a previous version of SQL-NS to the current version (available in both nscontrol and Management Studio)

All other administration operations (for example, registering an instance and enabling its components) can be invoked via methods on NMO classes. This section explains these methods and shows examples of their use.

Note

This section does not cover the use of NMO to update instances, because the NMO update process is based on concepts that are not introduced until the later section "Using NMO to Reflect the Contents of an Instance" (p. 555). Performing updates with NMO is covered near the end of this chapter, in the sidebar "Updating an Instance with NMO" (p. 563).

Registering an Instance with NMO

Registering an instance creates its Registry keys and performance counters, and, optionally, installs its Windows service. Using the standard tools, instances are registered with nscontrol register or the Register Instance command in Management Studio.

Using NMO, an instance is registered by calling one of the RegisterLocal() methods on its Instance object. Listing 16.6 shows the implementation of the Register() method of the StockBrokerNMO class, which calls the RegisterLocal() method on the Instance object created by the code shown earlier in Listing 16.4.

Listing 16.6. The Register() Method of the StockBrokerNMO Class

public class StockBrokerNMO
{

   ...
   public void Register()
   {
       if (AuthenticationMode.WindowsAuthentication ==
           serviceAuthenticationMode)

       {
           // Register with just the Windows username
           // and password.
           instance.RegisterLocal(
               serviceUserName,
               servicePassword);
       }
       else
       {
           // Register with both the Windows username
           // and password, as well as the SQL username
           // and password.
           instance.RegisterLocal(
               serviceUserName,
               servicePassword,
               serviceSqlUserName,
               serviceSqlPassword);

       }

     }
     ...
  }

One of two RegisterLocal() overloads is called, depending on the authentication mode the SQL-NS Windows service should be configured to use. The desired authentication mode is stored in the serviceAuthenticationMode private member variable by the StockBrokerNMO constructor. If this member variable specifies Windows Authentication, the code in Listing 16.6 calls the Instance object's RegisterLocal() method that takes a Windows username and password. This registers the instance and installs the service, configuring it to run with the specified Windows credentials. At runtime, the service will connect to the SQL Server using Windows Authentication.

If the service should use SQL Server Authentication, the code in Listing 16.6 calls an overload of RegisterLocal() that takes a SQL username and password, in addition to the Windows username and password. Again, the instance is registered and the service is configured to run under the given Windows credentials. But the supplied SQL username and password are also stored in the Registry (in an encrypted form). At runtime, the service uses this SQL username and password to connect via SQL Server Authentication.

Note

In addition to the two shown in Listing 16.6, the Instance class also provides other RegisterLocal() overloads, some of which don't install the SQL-NS service. When the service isn't needed (for example, when registering an instance on the machine that hosts only a subscription management interface), you can call one of these alternative overloads instead of the ones shown in Listing 16.6. See the reference for the Instance class in the SQL-NS Books Online for complete documentation on all variants of the RegisterLocal() method.

Besides installing the SQL-NS service and the performance counters, the main purpose of registering an instance is to record the name of the SQL Server that hosts the instance and application databases. At runtime, the SQL-NS engine (and parts of the SQL-NS API) locate the databases by looking up this SQL Server name in the Registry. We looked at how the SQL Server name is stored in the Registry when we first examined the registration process in Chapter 4, "Working with SQL-NS Instances," (see the "Registry Keys" section, p. 100).

When you register an instance with nscontrol register, you are required to specify the SQL Server name as one of the command-line arguments. But notice that the SQL Server name is not specified anywhere in the calls to the RegisterLocal() methods shown in Listing 16.6. This is because the SQL Server name is obtained from the Server object to which the Instance object is linked via its parent chain (recall that the parent of an Instance object is a NotificationServices object attached to a Server object).

Note

The Instance class also provides static overloads of the RegisterLocal() method that take the name of the SQL Server as an argument. You can use these static methods to register an instance created on any SQL Server, without creating an Instance object to represent it. This is possible because instance registration does not require a connection to the SQL Server. See the class reference for Instance in the SQL-NS Books Online for details on these static RegisterLocal() methods.

The name of the RegisterLocal() method was chosen to emphasize the fact that it registers the instance on the local machine, not the machine hosting the SQL Server to which the Instance object is linked. In other words, all Registry keys and performance counters are created only on the machine running the NMO code, regardless of where the SQL Server may be located.

Enabling an Instance with NMO

To enable a whole instance with NMO, you call its Instance object's Enable() method. This is shown in Listing 16.7, in the context of the StockBrokerNMO example.

Listing 16.7. The Enable() Method of the StockBrokerNMO Class

   public class StockBrokerNMO
   {

        ...
        public void Enable()
        {
            instance.Enable();
        }
        ...
   }

In addition to enabling a whole instance, NMO also supports enabling components of an instance selectively. The Instance class has methods, EnableSubscribers() and EnableSubscriptions(), which you can use to enable just subscriber and subscription management, respectively. Other components can be enabled via methods on their corresponding NMO classes. Specifically, the Application, HostedEventProvider, Generator, and Distributor classes all provide Enable() methods.

Calling the Enable() method on an Application object enables all components in the application it represents. The Application class also defines an EnableSubscriptions() method to enable only subscription management in the application. Calling the Enable() method on an object that represents a component of an application (for example, a Distributor object) enables only that component.

For all methods that enable components, NMO provides corresponding methods that disable the same components. Specifically, all classes that have Enable() methods also have Disable() methods. These methods behave symmetrically: whatever a particular class's Enable() method enables, its Disable() method disables. Similarly, methods such as EnableSubscribers() and EnableSubscriptions() have corresponding DisableSubscribers() and DisableSubscriptions() counterparts.

Deleting and Unregistering an Instance with NMO

NMO provides methods on the Instance class for unregistering and deleting an existing SQL-NS instance. Listing 16.8 shows these methods used in the cleanup code that runs at the start of the sample application. This code unregisters and then removes any existing copy of the StockBrokerNMO instance that may have been created by a previous run of the program.

Listing 16.8. The Cleanup() Method of the StockBrokerNMO Class

public class StockBrokerNMO
{

    ...
    public void Cleanup()
    {
        // Get the collection of existing instances.
        InstanceCollection existingInstances =
            server.NotificationServices.Instances;

       // Check if the StockBrokerNMO instance is contained
       // within the collection of existing instances.
       if (existingInstances.Contains(InstanceName))
    {
       // Get the instance object.
       Instance existingInstance =
           existingInstances[InstanceName];

       // Unregister the instance.
       existingInstance.UnregisterLocal();

      // Drop the instance.
      existingInstance.Drop();

    }

  }
  ...
}

The cleanup method first checks whether the instance exists by examining the collection of instances on the server's NotificationServices property. If this instance collection contains an object with our instance's name, the instance already exists. The Instance object for this existing instance is obtained from the instances collection, using the instance name as an index into the collection.

After the instance object is obtained, the code then calls its UnregisterLocal() method to remove the instance's registration. It then calls the Drop() method to remove the instance completely.

The Drop() method performs the same operations as nscontrol delete and the Delete Instance command in Management Studio: it deletes all database objects the SQL-NS compiler created for the instance. As such, it may take several minutes to complete.

Tip

The technique for obtaining an existing Instance object, shown in Listing 16.8, is a common pattern in NMO programming. Unlike the Create(), Register(), and Enable() methods, which are usually invoked on new Instance objects created to define a new instance, the UnregisterLocal() and Drop() methods are usually called on Instance objects that represent existing instances. This is also true of the Disable() method (explained briefly in the "Enabling an Instance with NMO" section, p. 552) and the Update() method (described in the sidebar "Updating an Instance with NMO," p. 563, later in this chapter). Whenever you need to call these methods, you'll probably use the standard pattern shown in Listing 16.8 to first obtain the existing Instance object.


Testing the StockBrokerNMO Sample Application

The StockBrokerNMO sample application takes a number of command-line arguments, including the target SQL Server name, the credentials of the service account, and the SQL usernames and passwords to use for creating and running the instance. The C:\SQL-NS\Samples\StockBrokerNMO\Scripts directory contains a batch file that invokes the StockBrokerNMO program and passes the appropriate values for the command-line arguments, based on the development environment configuration you established in Chapter 2, "Getting Set Up." Use the following instructions to build the StockBrokerNMO application, run it with the batch file, and then test the StockBrokerNMO instance it creates.

1.
If you haven't done so already, open the StockBrokerNMO Visual Studio solution using the instructions given in the section "An NMO Sample Application," (p. 534).

2.
Build the StockBrokerNMO solution in Visual Studio.

3.
Open a Notification Services Command Prompt and navigate to the C:\SQL-NS\Samples\StockBrokerNMO\Scripts directory.

4.
Invoke the run.cmd script. The script will prompt you for the password of the service account and, if you're using SQL Server Authentication, the passwords of the development and test SQL Server accounts. It then starts the StockBrokerNMO program, passing all the required command-line arguments. Several of the argument values are obtained from the environment variables established when you set up your development environment.

5.
Wait for the StockBrokerNMO program to complete. When it is finished, the StockBrokerNMO instance will have been created, registered, and enabled.

6.
Run the T-SQL script, C:\SQL-NS\Samples\StockBrokerNMO\Scripts\SQLPermissions.sql in Management Studio. This script grants the necessary permissions for the service to access the newly created instance database.

7.
Start the SQL-NS Windows service for the StockBrokerNMO instance using the following command:

net start NS$StockBrokerNMO

8.
Run the T-SQL script C:\SQL-NS\Samples\StockBrokerNMO\Scripts\Test.sql in Management Studio. This script enters some sample subscribers and subscriptions and then submits a set of test events.

9.
Observe the contents of the file C:\SQL-NS\Samples\StockBrokerNMO\FileNotifications.txt. Within 30 seconds, notifications should be written to this file.

As this sample shows, a SQL-NS instance created using the NMO API behaves exactly like one created via the standard SQL-NS tools. In fact, the StockBrokerNMO instance can now be managed like any other SQL-NS instance, using those standard tools.

When you're done working with the StockBrokerNMO sample instance, you can remove it from your system by running the script cleanup.cmd from C:\SQL-NS\Samples\StockBrokerNMO\Scripts.


Using NMO to Reflect the Contents of an Instance

In the preceding sections, you saw the NMO API used to create and administer a new SQL-NS instance. The NMO API can also be used to reflect over an existing instance. Reflection allows you to programmatically interrogate an existing SQL-NS instance to discover its contents. Think of reflection as the inverse operation of creating an instance: rather than defining a new instance by creating new NMO objects, your code can examine an existing instance by obtaining the corresponding NMO Instance object and then reading its properties and the properties of its child objects.

The reflection capabilities of the NMO API are useful in building dynamic management tools for SQL-NS instances. A good example is the SubscriptionLoader tool we used in Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State," to enter subscriptions into the music store sample application. The SubscriptionLoader tool presents a user interface by which all the properties of a subscription can be specified, including the values of the subscription fields. When you used it with the music store sample application, you used controls in the SubscriptionLoader user interface to select among the application's two subscription classes. The subscription fields grid was populated with a line for each of the chosen subscription class's fields.

Despite the appearance of being tailored to the music store instance, the SubscriptionLoader user interface is in fact generic. The names of the music store instance's application, its subscription classes, and their subscription fields are not coded into the interface. SubscriptionLoader can work against any SQL-NS instance because it uses the NMO API to dynamically discover the list of applications and subscription classes, and the fields in those subscription classes. It then uses this information to populate its user interface dynamically.

This section explains the use of the NMO API for reflecting SQL-NS instances. As a practical example, we examine the actual SubscriptionLoader code.

The SubscriptionLoader Code

A Visual Studio solution containing the SubscriptionLoader source code is located in the C:\SQL-NS\Tools\Source\SubscriptionLoader directory. The solution file is called SubscriptionLoader.sln. Open this solution in Visual Studio and refer to it as you read through this section.

Most of the SubscriptionLoader code deals with creating and maintaining the program's user interface. In this section, we look only at the parts of this code that use the NMO API.

Open the file MainForm.cs from the Solution Explorer. All the code covered in this section is in this file. By default, Visual Studio may open MainForm.cs in design view, showing the layout of the visual controls in the program's user interface. As you read through this section, you'll need to look at the code, not the user interface design, so you should switch from design view to code view. To do this, right-click MainForm.cs in the Solution Explorer and choose View Code.

Caution

Opening MainForm.cs in design view before the solution is built for the first time may result in the following error message from the designer: Could not find type 'SubscriptionLoader.SubscriptionFieldData'. If you build the solution at least once, close the file, and open it again, you should no longer see this error message. However, because you don't need to look at the file in design view for the purposes of this chapter anyway, you can ignore this message if you see it. Open the file in code view, as described in the preceding paragraph.

Preparing to Reflect: Connecting to a Server

Reflection begins with an SMO Server object representing a SQL Server. From the Server object, the SQL-NS instances on the SQL Server can be enumerated and their properties discovered.

In the SubscriptionLoader program, the connection to the SQL Server is established in the method MainForm_Load() in MainForm.cs. The MainForm_Load() method is called when SubscriptionLoader is started. Its purpose is to initialize the state of the application, including its user interface controls. The relevant parts of the MainForm_Load() implementation are shown in Listing 16.9.

Listing 16.9. The MainForm_Load() Method in the SubscriptionLoader's MainForm Class

...
using Microsoft.SqlServer.Management.Common;
using Microsoft.SqlServer.Management.Smo;
using Nmo=Microsoft.SqlServer.Management.Nmo;
...
namespace SubscriptionLoader
{
    public partial class MainForm : Form
    {
        private ServerConnection connection = null;
        ...
        private Nmo.NotificationServices ns = null;

        ...

        private void MainForm_Load(object sender, EventArgs e)
        {

           // Show the database connection dialog.
           DBConnectionDialog connectDlg = new DBConnectionDialog();

           if (connectDlg.ShowDialog(this) == DialogResult.OK)
           {
               // Create a connection object from the information
               // supplied in the connection dialog.
               connection = new ServerConnection();

               connection.ServerInstance = connectDlg.ServerName;

               if (AuthenticationMode.SqlServerAuthentication ==
                   connectDlg.AuthenticationMode)

               {
                   connection.LoginSecure = false;
                   connection.Login = connectDlg.SqlLoginName;
                   connection.Password = connectDlg.SqlPassword;

                }

                ...

                // Use the connection to obtain a Server object.
                try
                {
                    Server server = new Server(connection);
                    ns = server.NotificationServices;

                 }

                 catch (Exception ex)
                 {

                      ...
                  }

                  // We should have obtained a reference to the
                  // NotificationServices object on the server
                  // at this point.
                  Debug.Assert(null != ns);

                  PopulateInstances();
               }
               ...
            }
        }
    }

As Listing 16.9 shows, MainForm_Load() opens a database connection dialog box in which the user can specify a SQL Server, select an authentication mode, and provide credentials if SQL Server Authentication is selected. After this information is obtained from the connection dialog box, the code establishes a connection to the specified SQL Server by creating a ServerConnection object. It then uses this to instantiate a Server object. A reference to the Server object's NotificationServices property is stored in the private member variable ns for use in other methods.

Finally, the MainForm_Load() method calls the helper method PopulateInstances(). This method fills one of the drop-down lists in the program's user interface with the names of the instances that exist on the selected SQL Server. We look at the implementation of this method in the following section.

Note

In the file MainForm.cs, the NMO namespace, Microsoft.SqlServer.Management.Nmo, is assigned the alias Nmo. This is accomplished by the using directive near the top of the file:

using Nmo=Microsoft.SqlServer.Management.Nmo;

Notice the Nmo= in front of the namespace name. Because the namespace is declared in this way, the token Nmo can be used as shorthand for the entire namespace name. All NMO class names are referenced in the file with this namespace prefix. For example, the declaration of the private member variable that stores an Instance object is given as

private Nmo.Instance selectedInstance = null;

Because the earlier using directive establishes the Nmo alias, Nmo.Instance is equivalent to using the fully qualified class name Microsoft.SqlServer.Management.Nmo.Instance.

Enumerating SQL-NS Instances

The SQL-NS instances that exist on a particular SQL Server are represented by a collection of NMO Instance objects. This collection is accessed via the Instances property on the NotificationServices object attached to the corresponding Server object. By iterating over this collection, the SQL-NS instances can be enumerated. In this section, we look at the part of the SubscriptionLoader code that enumerates instances this way.

The SubscriptionLoader user interface provides drop-down lists (represented as combo box controls) with which users can select a SQL-NS instance, an application within the selected instance, and then a subscription class within that application. The combo box controls are populated by code that uses NMO classes to enumerate instances, applications, and subscription classes. Listing 16.10 shows the PopulateInstances() method, which populates the first combo box with instance names.

Listing 16.10. The PopulateInstances() Method in the SubscriptionLoader's MainForm Class

public partial class MainForm : Form
{

   ...
   private void PopulateInstances()
   {

       comboBoxInstanceName.Items.Clear();
       foreach (Nmo.Instance instance in ns.Instances)
       {
          comboBoxInstanceName.Items.Add(instance.Name);
       }

       ...
   }
   ...
}

PopulateInstances() first clears any existing elements in the combo box before filling it with the names of the current SQL-NS instances on the SQL Server specified earlier. Recall from Listing 16.9 that the NotificationServices object associated with the selected server was saved in the private member variable, ns. This is used here to obtain the Instances collection. The foreach loop iterates over the Instance objects in this collection. The code in the loop's body adds the name of each instance (obtained from the Name property of the Instance object) to the combo box. Thus, by walking through the Instances collection on the NotificationServices object, all the instances are enumerated.

When a user selects an instance name in the instance name combo box, the handler, comboBoxInstanceName_SelectedIndexChanged(), is called. Some of the code in this handler is shown in Listing 16.11.

Listing 16.11. The comboBoxInstanceName_SelectedIndexChanged() Method in the SubscriptionLoader's MainForm Class

public partial class MainForm : Form
{
   ...
   private Nmo.Instance selectedInstance = null;
   ...
   void comboBoxInstanceName_SelectedIndexChanged(
       object sender,
       EventArgs e)
   {
       string selectedInstanceName =
           (string)comboBoxInstanceName.SelectedItem;
       selectedInstance = ns.Instances[selectedInstanceName];

       ...

       // Populate the selection controls.
       PopulateApplications();
       ...
   }
   ...
}

The name of the selected instance is read from the combo box and stored in the local string variable, selectedInstanceName. This is then used as an index into the Instances collection on the stored NotificationServices object, ns. The Instance object representing the selected instance is returned. This is stored in the selectedInstance private member variable, for use in other methods. After the Instance object for the selected instance is obtained and stored, the PopulateApplications() method is called to populate the user interface controls with the list of applications in the selected instance. The next section looks at the code that accomplishes this.

Enumerating the Contents of an Instance

When an Instance object that represents an existing instance is obtained, its properties are initialized with values that reflect the real attributes of the instance, as read from the configuration in the database. Child objects are also created to represent the existing components of the instance (for example, its delivery channels, protocols, and applications, as well as the event, subscription, and notification classes within those applications). These objects are linked in parent-child relationships and ultimately attached to the Instance object. This forms a hierarchy that correctly represents the structure of the existing instance. This is the same kind of hierarchy that the code in Listings 16.4 and 16.5 created to define a new instance with NMO objects; the child objects are of the same NMO classes you saw in those earlier listings.

Note

Conceptually, you can think of the entire hierarchy being populated when the Instance object is first obtained. In reality, some of the child objects are constructed only when they are first accessed. This is merely a performance optimization and does not affect the way the API is used.

To enumerate the entities within an instance, we start with an existing Instance object (obtained as shown in the previous section) and traverse its various child objects. Continuing down the parent-child hierarchy, we can navigate the properties and collections on these child objects to learn about their children. In this way, we can, for example, go through an Instance object's Applications collection to find a particular Application object, and then through the SubscriptionClass objects in that Application object's SubscriptionClasses collection. We can continue all the way down to the SubscriptionField objects in a SubscriptionClass object to learn the name and data type of each field. This is, in fact, what the SubscriptionLoader tool does. Listing 16.12 shows some of the related code.

Listing 16.12. Methods in the SubscriptionLoader's MainForm Class That Enumerate Applications and Subscription Classes

public partial class MainForm : Form
{
    ...
    private Nmo.Instance selectedInstance = null;
    private Nmo.Application selectedApplication = null;
    private Nmo.SubscriptionClass selectedSubscriptionClass = null;
    ...
    private void PopulateApplications()
    {
       comboBoxApplicationName.Items.Clear();
       foreach (Nmo.Application application in
           selectedInstance.Applications)

       {
           comboBoxApplicationName.Items.Add(application.Name);
       }

       ...

     }

     void comboBoxApplicationName_SelectedIndexChanged(
          object sender,
          EventArgs e)
     {
          // Establish a connection to the selected application.
          string selectedApplicationName =

          (string)comboBoxApplicationName.SelectedItem;
      selectedApplication =
           selectedInstance.Applications[selectedApplicationName];

      ...
      PopulateSubscriptionClasses();
    }

    private void PopulateSubscriptionClasses()
    {
        comboBoxSubscriptionClass.Items.Clear();
        foreach (Nmo.SubscriptionClass subscriptionClass in
            selectedApplication.SubscriptionClasses)

      {

            comboBoxSubscriptionClass.Items.Add(
                subscriptionClass.Name);
      }

      ...

    }

    void comboBoxSubscriptionClass_SelectedIndexChanged(
        object sender,
        EventArgs e)
    {

        string selectedSubscriptionClassName =
             (string)comboBoxSubscriptionClass.SelectedItem;
        selectedSubscriptionClass =
              selectedApplication.
SubscriptionClasses[selectedSubscriptionClassName];

        ...

        // Populate list of subscription fields.
        PopulateSubscriptionData();

    }
    ...

  }

The PopulateApplications() method populates the applications combo box in the user interface with the list of applications within the selected instance. Its structure is much the same as the PopulateInstances() method we looked at in Listing 16.10. First, the applications combo box is cleared. Then, the code iterates over the Application objects in the Applications collection on the selectedInstance object. Recall from Listing 16.11 that selectedInstance is set to the Instance object representing the selected instance. The Applications collection on the Instance object contains one Application object for each application in the instance. By walking through the contents of this collection, the code enumerates all applications, adding the name of each one to the applications combo box.

When a particular application name is selected in the user interface, the comboBoxApplicationName_SelectedIndexChanged() handler is called. The code in this handler obtains the Application object corresponding to the selected application. It does this by using the selected application name (read from the combo box control) as an index into the Instance object's Applications collection. The Application object obtained from the collection is stored in the selectedApplication private member variable.

Following essentially the same patterns, the code in the PopulateSubscriptionClasses() method populates the list of subscription classes, and the comboBoxSubscriptionClass_SelectedIndexChanged() handler reacts to the selection of a particular subscription class by obtaining and storing the corresponding SubscriptionClass object. Notice that the subscription classes in the application are enumerated via the SubscriptionClasses collection on the Application object.

In code not shown in Listing 16.12, the application obtains the names of the fields in the selected subscription class by enumerating the SubscriptionField objects in the SubscriptionClass object's SubscriptionFields collection. Reading the name of each subscription field from the SubscriptionField object's Name property, the application populates the field name labels in the data grid used to enter field values.

Although only a few examples are shown here, the NMO API has the capability to enumerate all the entities within an instance and its applications. By traversing all the properties and child objects, starting from an Instance object, the complete instance definition can be obtained. Listings 16.4 and 16.5 show more of the NMO classes and properties in the context of defining a new instance, but these same classes and properties are used to reflect existing instances as well. The organization of the NMO classes is shown in Figure 16.1 (although the collection classes are not explicitly labeled). The SQL-NS Books Online provides complete reference documentation on each one.

Updating An Instance With Nmo

The NMO API can be used to update the definition of an existing SQL-NS instance. The same update functionality provided by the standard update tools, nscontrol update and the Update Instance command in Management Studio, is provided through NMO. This chapter's source code does not include an example of updating an instance, but this sidebar provides an overview of the process.

Updating an instance with NMO involves a combination of the techniques used to reflect over an existing instance and those used to create new instances. To begin, you obtain an Instance object that represents the instance you want to modify. As you saw in the reflection examples, you can then obtain information about the instance's subcomponents by navigating the NMO hierarchy. What the reflection examples didn't show is that, in addition to just reading the structure of the instance, you can also modify it.

You can add new components by creating new NMO objects and linking them into the existing object hierarchy (just like you did when defining a new instance). For example, to add a new notification class, you create a new NotificationClass object, set its parent to be an existing Application object, and then add it to the parent's NotificationClasses collection.

To modify existing components, you change the property values on the existing NMO objects. Some properties, such as the names of objects, cannot be changed. Attempting to change the values of these properties results in an exception.

You can remove existing components by extricating them from the existing object hierarchy. To remove a component, you remove the corresponding object from its parent object's collection. For example, to remove an event class, you locate its EventClass object and then pass it to the Remove() method on the parent Application object's EventClasses collection.

After making the necessary changes to the objects that represent the instance, you call the Instance object's Update() method to compile the changes and apply them to the database objects. Doing this invokes the SQL-NS compiler, which performs the same operations it does when invoked from nscontrol update or the Update Instance command in Management Studio. It compares the new instance definition (expressed in the modified NMO objects) with the existing compiled definition in the database and then updates the database objects according to the differences found. Until the Update() method is called, any changes made to the instance definition in the NMO objects in memory are not reflected in the database objects.

Before calling the Update() method on the Instance object, the instance must be disabled. You can disable the instance programmatically using the Disable() method on the Instance object, or you can use the standard SQL-NS tools to disable the instance from the command line or Management Studio.

As you've probably noticed from running the standard update tools, the SQL-NS compiler can take a significant amount of time to complete an update. Therefore, the Update() method can take a long time to return and will block the calling thread for the duration of the update process.


Summary

This chapter introduces the Notification Services Management Objects (NMO) API, which provides programmatic access to most SQL-NS management operations. Using the NMO API, you can write code that creates new SQL-NS instances, administers existing instances, and reflects over the entities they contain. The capabilities of the NMO API are most useful for defining new SQL-NS instances dynamically and writing generic management tools that adapt to the structure of any SQL-NS instance.

The first part of this chapter looks at the classes in the NMO API. These classes are organized into a hierarchical structure, based on parent-child relationships. The classes in the API encapsulate the same information that can be specified in XML ICFs and ADFs.

To create a new SQL-NS instance using the NMO API, you construct the instance definition by instantiating NMO classes, filling in their properties, and linking them in the appropriate parent-child relationships. When the instance is completely defined, you call the Create method on its Instance object to invoke the SQL-NS compiler. If the instance definition is valid, the compiler creates the corresponding instance and application database objects. In this chapter, we used NMO to create a new SQL-NS instance called StockBrokerNMO. This new instance is essentially the same as the StockBroker instance from Chapter 3, except that it is created programmatically with the NMO API, rather than from an ICF and ADF.

Many SQL-NS administration tasks can be invoked programmatically, via methods on the NMO Instance class. This chapter covers the methods used to register, enable, disable, update, unregister, and delete a SQL-NS instance with the NMO API. The code examples in this chapter show these methods used in the context of the StockBrokerNMO sample instance.

The final section of this chapter looks at the use of the NMO API to reflect over existing instances. Using reflection, the complete definition of an instance can be obtained programmatically. Reflection code uses the same classes that are used to define new instances. Starting from the Instance object that represents an existing instance, the instance definition is obtained by navigating the various properties and child objects. The code examples in this chapter illustrate how the NMO API's reflection capabilities are used in the SubscriptionLoader tool, introduced in Chapter 6.


Chapter 17. Hosting the SQL-NS Execution Engine

IN THIS CHAPTER

  • Using the SQL-NS Hosting API

  • EngineHost: A Sample Hosting Application

In every example presented in this book so far, the SQL-NS engine runs in a Windows service. To start and stop the engine, you start and stop the service. Although this is adequate in most scenarios, there are cases in which it's desirable to run the SQL-NS engine as an integrated part of a larger application, rather than as a separate service.

To address these cases, SQL-NS provides a Hosting API. This API offers facilities for running the SQL-NS engine within a custom application. When invoked via the Hosting API, the engine and all its subcomponents execute within the hosting application's processno service is involved. The functionality of the engine does not change as a result of being hosted this way. Its behavior (and that of the underlying SQL-NS instance it serves) is the same as when it runs in the Windows service.

Hosting the SQL-NS engine helps integrate SQL-NS functionality into larger applications that provide more than just notification capabilities. Often, such applications already have one or more services (or other processes that run continuously). Hosting the SQL-NS engine may reduce the administrative overhead in these applications: rather than managing the SQL-NS engine separately as a Windows service, it can be embedded in one of the application's existing processes. This chapter explains the SQL-NS Hosting API and shows its use in a sample application.


Using the SQL-NS Hosting API

Building an application that hosts the SQL-NS engine is exceedingly simple. In the hosting application's code, you use the SQL-NS Hosting API to instantiate, start, and stop the engine. Between starting and stopping, the hosted engine manages its own resources, including the threads on which it executes, so nothing else is needed in the hosting application's code.

The SQL-NS Hosting API consists of exactly one classand it's one you've already seen: NSInstance. Previous chapters showed the NSInstance class used to encapsulate the properties needed to connect to a SQL-NS instance's database. But the NSInstance class has other members, not shown in previous chapters, that relate to hosting the SQL-NS engine. Specifically, NSInstance provides a StartInstance() method to start the engine, a StopInstance() method to stop the engine, an IsRunning property used to query the state of the engine, and an Error event that communicates fatal errors from the engine. This section describes the general usage of these NSInstance members. The following section shows a practical example of their use in a sample hosting application.

Note

Don't confuse the NSInstance class, which is part of the standard SQL-NS API, with the Instance class that is part of NMO. NSInstance was first introduced in Chapter 7, "The SQL-NS Subscription Management API," to represent connections to the instance database. The Instance class, covered in Chapter 16, "Using Notification Services Management Objects," fully encapsulates all the properties that define an instance.

Creating and Starting a Hosted Engine

Hosting begins with the creation of an NSInstance object. In the NSInstance constructor, you specify an instance name; the resulting NSInstance object can then be used to host the engine for that instance. Because the NSInstance constructor reads configuration information from the instance's Registry key, the instance must have been previously registered. The NSInstance constructor throws an exception if the specified instance isn't registered on the local machine.

Note

When registering an instance, you need not install the SQL-NS Windows service if the engine will only be hosted in a custom application. However, installing the service does no harm. The engine can still be hosted, even if the service is installed. All three registration mechanisms (nscontrol register, Management Studio, and NMO) provide options for registering an instance without installing its Windows service.

After creating an NSInstance object, you call its StartInstance() method to start the engine. Internally, StartInstance() creates a separate thread of execution on which the engine's startup operations are performed. This means that the StartInstance() method returns fairly quicklyit does not block waiting for the engine to start.

On startup, the engine carries out the same operations it does when hosted in the SQL-NS service: it connects to the databases and determines which SQL-NS components should run on the local computer. It then begins the execution of those components, creating and managing threads as needed. The only difference in a hosted scenario is that these components run in the hosting application's process, rather than in the SQL-NS service.

Note

The Hosting API wraps the same engine implementation used in the SQL-NS Windows service. In fact, the SQL-NS Windows service can be thought of as another hosting application. Internally, the SQL-NS Windows service uses the Hosting API to start and manage the engine.

Hosting opens up the possibility of several copies of the same SQL-NS engine being started concurrently on a single machine. For example, a hosting application could be started two (or more) times in parallel. Or a hosting application and the SQL-NS Windows service could be started at the same time. Even two threads in the same hosting application process could each create and start an engine for the same SQL-NS instance.

Having more than one engine for the same instance running on the same machine would obviously cause problems. Each one would try to run the same components, leading to conflicts and possible corruption of the instance state stored in the database. To prevent this, the hosting API explicitly blocks multiple copies of the same engine from running on the same machine. Using machinewide synchronization mechanisms, the API ensures that only the first engine started for a given instance is allowed to run. Subsequent attempts to start the same instance's engine (whether in another hosting process, the SQL-NS Windows service, or another thread in the same process) will fail.

Note

The restriction that only one copy of an engine may run at a time applies only to a single instance and a single machine. Engines for different instances may run concurrently on a single machine. Also, different machines can each run a copy of the same instance's engine concurrently. This is in fact needed to support the distributed scale-out scenario described in the section "Scaling Out the SQL-NS Engine" (p. 454) in Chapter 13, "Deploying a SQL-NS Instance." On each machine, the engine runs only the components configured to run on that machine.

Across instances and machines, you can mix and match hosting applications with the SQL-NS Windows service. In other words, on one machine in a distributed deployment, the engine can run in a custom hosting application, and on another machine, it can run in the Windows service. On the same machine, the engine for one instance can run in a hosting application and the engine for another instance can run in the Windows service.

The IsRunning property of an NSInstance object returns a Boolean value that indicates whether the engine is currently running. However, it's important to note that the check performed by the IsRunning property is limited to the specific NSInstance object on which it is accessed. IsRunning returns true only if the running engine was started by calling that particular object's StartInstance() method. If the engine is running but was started via another NSInstance object or the Windows service, IsRunning returns false.

Authentication and Security Issues in Hosting the SQL-NS Engine

A hosted SQL-NS engine can connect to the database via Windows Authentication or SQL Server Authentication. With Windows Authentication, the engine connects using the credentials of the account under which the hosting application runs. Alternatively, a SQL username and password can be used for SQL Server Authentication.

The choice of authentication mode is determined by the way the NSInstance object is constructed in the hosting application's code. If the NSInstance constructor is passed only an instance name, the engine connects via Windows Authentication. If the constructor is passed a SQL username and password, in addition to the instance name, the engine uses SQL Server Authentication.

Caution

When hosted in a custom application, the SQL-NS engine never reads SQL credentials from the SQL-NS instance's Registry key. Even if a SQL username and password are supplied at registration time and stored in the Registry, they are not made available to the hosted engine. Credentials stored in the instance's Registry key are available only to the SQL-NS Windows service.

This means that if you use SQL Server Authentication, you must manage the SQL username and password for the engine on your own and supply them to the NSInstance constructor. If your application stores credentials, it's strongly recommended that you use the Windows Data Protection API (DP API) to encrypt them. Documentation for the DP API is available in MSDN or online at http://msdn.microsoft.com/library/en-us/dnsecure/html/windataprotection-dpapi.asp.

Regardless of the SQL authentication mode you choose, it's good practice to always run the SQL-NS engine with the fewest possible privileges. When the engine runs in the SQL-NS service, this is accomplished by configuring the service to run as a low privileged account to which you selectively grant only the permissions needed. When hosted, the engine runs in the security context of the Windows account under which the hosting application runs. Always try to keep the permissions granted to this account to a minimum. Hosting the engine in an application that requires high levels of privilege may introduce security vulnerabilities.

Stopping the Hosted Engine

The StopInstance() method of the NSInstance class stops a hosted SQL-NS engine. Hosting applications save a reference to the NSInstance object on which they call StartInstance() and then use this to call StopInstance() as needed.

StopInstance() succeeds only when called from the same process that started the instance via StartInstance(). Calling StopInstance() in any other process results in an exception. This enforces the concept that only one process manages the SQL-NS engine for a given instance at a time (on a single machine). You cannot, for example, create an NSInstance object and call its StopInstance() method to stop an engine started by another process.

A hosted engine is automatically stopped if its NSInstance object is garbage collected. Therefore, a hosting application should always keep at least one reference to each NSInstance object on which it calls StartInstance(), to prevent these objects from being prematurely garbage collected. After calling StopInstance(), all references can be released, allowing normal garbage collection to proceed.

Receiving Error Events from the Hosted Engine

During the course of a SQL-NS engine's execution, it may encounter a fatal error that causes it to stop. Applications that host the engine may need to react to such errors by displaying a message to the user, attempting to restart the engine, or taking some other corrective action.

The NSInstance class exposes an event called Error that is raised whenever a fatal error occurs during the engine's execution. Hosting applications can attach an event handler delegate to this event. Whenever the event is raised, the event handler is called and can deal with the error by taking the appropriate action.

The delegate for NSInstance.Error event handlers is declared as follows:

public delegate void ErrorEventHandler(object sender, ErrorEventArgs e);

In the hosting application's code, you declare an event handler with this signature, create a delegate to the handler, and attach it to the Error event on an NSInstance object. It's important to note that when the event is raised, the event handler is called from the SQL-NS engine's execution thread. This is always a different thread from the one on which the NSInstance object's StartInstance() method was called. Therefore, any operations done inside the event handler need to be thread-safe.

The Error event is raised only for errors that cause the engine to stop. Other nonfatal engine errors, such as failures to deliver notifications, are not communicated through this event. Nonfatal errors are written to the Application Event Log, just as they are when the engine runs in the Windows service.

As explained in the "Setting the Logging Level" section (p. 486) in Chapter 14, "Administering a SQL-NS Instance," the amount of information written to the Application Event Log is controlled by settings in the NSService.exe.config file when the engine runs in the SQL-NS Windows service. When hosting the engine in a custom application, you can control the logging level with a similar configuration file.

The name of the configuration file must be the name of the hosting application's executable file, with .config appended. For example, if the hosting application's executable is called EngineHost.exe, its configuration file must be called EngineHost.exe.config. The configuration file must be placed in the same directory as the executable. Listing 17.1 shows the configuration file elements that control the SQL-NS engine's logging levels. You can include these elements in the configuration file for any hosting application, to control the information the hosted engine writes to the event log.

Listing 17.1. Configuration File Settings That Control the SQL-NS Logging Level

<configuration>
    ...
    <system.diagnostics>
        <switches>
            <add name="LogAdministrative" value="2"/>
            <add name="LogService" value="2"/>
            <add name="LogEventProvider" value="2"/>
            <add name="LogEventCollector" value="2"/>
            <add name="LogGenerator" value="2"/>
            <add name="LogDistributor" value="2"/>
            <add name="LogVacuumer" value="2"/>
            <add name="LogOther" value="2"/>
            <add name="LogPerformanceMonitor" value="2"/>
        </switches>
    </system.diagnostics>
    ...
</configuration>

The valid values for each log switch are explained in Chapter 14 (in the "Setting the Logging Level" section, p. 486). If a hosting application does not have a configuration file, or if the configuration file does not contain settings for the SQL-NS log switches, the hosted engine will not log any information to the Application Event Log. Therefore, you should always create a configuration file for your hosting applications and specify at least a minimal logging level. In most cases, a logging level of 2 is sufficient. (This is the default for the SQL-NS service.)


EngineHost: A Sample Hosting Application

This section introduces EngineHost, a new sample application. EngineHost is a commandline program that hosts the engine for any SQL-NS instance. It takes the instance name as a command-line argument, starts the engine, and waits for the user to press a key before stopping the engine. If a fatal error occurs while the engine is running, the program displays an error message and then quits.

EngineHost doesn't do anything other than host the SQL-NS engine. As such, it doesn't offer much beyond the standard SQL-NS Windows service, but its source code illustrates how the SQL-NS Hosting API is used. The same techniques and coding patterns used in the EngineHost sample can be used in a real hosting application.

The EngineHost Visual Studio Solution

The Visual Studio solution containing the EngineHost code is located at C:\SQL-NS\Samples\EngineHost. The solution file is called EngineHost.sln. Open this solution in Visual Studio now and use it to refer to the source code as you read through this section.

The Visual Studio Solution Explorer shows the various files that make up the EngineHost source code. The following two files relate to the SQL-NS Hosting API:

  • Program.cs

  • App.config

Program.cs contains the main driver code for the application, including the calls to the Hosting API. We examine parts of this code in the following sections. App.config is a configuration file that specifies settings for the SQL-NS engine's log switches, as shown in Listing 17.1. At build time, App.config is copied to the output directory in which the program's executable file is created, and it is renamed EngineHost.exe.config. The configuration settings specified in this file are read by the engine at runtime.

Use of the Hosting API in the EngineHost Code

The calls to the SQL-NS Hosting API in the EngineHost source code are made in the file Program.cs. This file contains the main driver code for the application, in the Program class. Listing 17.2 shows some of the code in the Program class's Main() method and some of its private member variables.

Listing 17.2. Use of the Hosting API in the EngineHost Program

[View full width]
class Program

{

    ...
    private static string instanceName;
    private static AuthenticationMode authenticationMode;
    private static string sqlUserName;
    private static string sqlPassword;
    private static NSInstance nsInstance = null;

    static int Main(string[] args)
    {
        // Assume failure code - set this to success at the end.
        int returnCode = 1;

        ...

        // Declare and initialize flag that tracks whether we
        // successfully started the instance.
        bool instanceStarted = false;
        try
        {
            // Create the NSInstance object.
            if (AuthenticationMode.WindowsAuthentication ==
                authenticationMode)
            {
                nsInstance = new NSInstance(instanceName);
            }
            else
            {
                nsInstance = new NSInstance(
                    instanceName,
                    sqlUserName,
                    sqlPassword);
            }

            // Attach the error handler delegate to the error event.
            nsInstance.Error +=
                new NSInstance.ErrorEventHandler(nsInstance_Error);

            // Start the instance.
            nsInstance.StartInstance();
            instanceStarted = true;

            Console.WriteLine(
                "Notification Services Engine for {0} instance has been started. Press any
   key to stop.",
                instanceName);

            // Wait for the user to stop the program.
            Console.ReadKey();
        }
        catch (NSException nsEx)
        {
            // Display an error message that includes
            // the Notification Services error code.
            Console.WriteLine("A Notification Services error has occurred.");
            Console.WriteLine(
                "Notification Services Error Code: {0}",
                (int)nsEx.ErrorCode);
            Console.WriteLine("Message: {0}", nsEx.Message);
            goto Exit;
        }
        catch (Exception ex)
        {
            // Display a generic error message.
            Console.WriteLine("An error has occurred.");
            Console.WriteLine(
                "Exception type: {0}",
                ex.GetType().Name);
            Console.WriteLine("Message: {0}", ex.Message);
            goto Exit;
        }
        finally
        {
            // Check that NSInstance isn't null and that we
            // did actually succeed in starting the engine.
            if (null != nsInstance && instanceStarted)
            {
                // Stop the engine.
                nsInstance.StopInstance();
            }
        }

        // Everything succeeded, so set success return code.
        returnCode = 0;

    Exit:
        return returnCode;
    }
...
}

The EngineHost program takes up to four command-line arguments. Two of these are required: an instance name and an authentication mode. The other two arguments are optional: they are used to specify a SQL username and password, if SQL Server Authentication is the chosen authentication mode. (The following section, "Testing the EngineHost Program," explains the program's command-line syntax in detail.)

The command-line arguments are processed and their values are stored in the private member variables instanceName, authenticationMode, sqlUserName, and sqlPassword. The code that validates and processes the command-line arguments is not shown in Listing 17.2 (although you can see this code in the Visual Studio solution). When looking at the code in Listing 17.2, assume that these private member variables contain the values of the command-line arguments.

The Main() method code shown in Listing 17.2 begins by declaring and initializing a Boolean flag that records whether the engine has been successfully started. This flag is checked in the cleanup phase of the program, to determine whether the engine should be stopped. The flag is initialized to false.

Next, the program creates an NSInstance object, using the appropriate constructor for the authentication mode specified in the command-line arguments. In the case of Windows Authentication, the constructor is passed just the instance name. In the case of SQL Server Authentication, the constructor takes the SQL username and password, along with the instance name.

After creating the NSInstance object, the code connects an error event handler to the object's Error event. The event handler is the method, nsInstance_Error(); its code is shown later, in Listing 17.3. In the code in Listing 17.2, an ErrorEventHandler delegate to this method is created and attached to the Error event.

After attaching the event handler delegate, the StartInstance() method is called on the NSInstance object. If StartInstance() returns without throwing an exception, the instanceStarted flag is set to true. This will cause the cleanup code to call the StopInstance() method later, to stop the engine.

The program then displays a message indicating that the engine has been started. Thereafter, it waits for keyboard input by calling the blocking method, Console. ReadKey(). This method returns only after the user presses a key. While waiting for a keypress, the SQL-NS engine continues to run on its own threads, processing events and generating and delivering notifications.

When Console.ReadKey() returns (after the user presses a key), control falls through to the finally block, which performs cleanup for the application. The code in the finally block checks that the NSInstance object isn't null and that the flag indicates the engine was started. If both conditions are true, it calls the NSInstance object's StopInstance() method to stop the engine's execution.

Listing 17.3 shows the code in the nsInstance_Error() event handler, which is called whenever the Error event is raised by the NSInstance object created in Listing 17.2.

Listing 17.3. Handler for the NSInstance.Error Event

class Program
{
    ...
    static void nsInstance_Error(
        object sender,
        System.IO.ErrorEventArgs e)
    {
        // Display an overall error message.
        Console.WriteLine(
            "A fatal error occurred during the engine's execution.");

        // Get the exception object.
        Exception ex = e.GetException();

        // If the exception is an NSException, show the NS

        // error code.
        if (ex is NSException)
        {
            NSException nsEx = ex as NSException;

            Console.WriteLine(
                "Notification Services Error Code: {0}",
                (int)nsEx.ErrorCode);
         }

         // Write the exception message.
         Console.WriteLine("Message: {0}", ex.Message);

         // This event is only fired for fatal errors, meaning the
         // engine has stopped. Alert the user.
         Console.WriteLine("The engine has stopped.");
    }
    ...
}

The first line of the event handler prints a message indicating that a fatal error occurred. Next, the exception object that encapsulates the fatal error is obtained by calling the GetException() method on the ErrorEventArgs object passed in. If the exception object turns out to be of the SQL-NS-specific NSException class, the event handler prints the SQL-NS error code. It then prints the error message from the exception object and a message indicating that the engine has stopped.

Together, Listings 17.2 and 17.3 include all the code required to host the SQL-NS engine. As you can see, not much code is involved.

The following section explains how to test the EngineHost program on your system, to see its hosting capabilities in action. Although this sample application doesn't have any functionality beyond hosting the engine and printing status and error messages, it illustrates the fundamental coding patterns common to most hosting applications. These same patterns can be applied in a real application that may perform other useful functions while hosting the SQL-NS engine.

Testing the EngineHost Program

To test the EngineHost program, we can use it to host the SQL-NS engine for the StockBrokerNMO instance defined in the previous chapter. Before you invoke EngineHost (as described in this section), first make sure the StockBrokerNMO instance has been created, enabled, and registered on your system. To do this, refer back to the instructions in the section "Testing the StockBrokerNMO Sample Application" (p. 554) in Chapter 16, and complete steps 1 through 6. Do not execute step 7, which starts the SQL-NS Windows servicewe will use the EngineHost program to host the SQL-NS engine instead.

Caution

If the StockBrokerNMO instance was previously created on your system and the service is currently running, stop it before continuing with the rest of the instructions in this section. You can stop the StockBrokerNMO instance's service with the following command:

net stop NS$StockBrokerNMO

If you have not already done so, build the EngineHost program from its Visual Studio solution. (Instructions for opening the solution are provided in the earlier section "The EngineHost Visual Studio Solution" p. 573.) After successfully building, you can run EngineHost by invoking EngineHost.exe from the C:\SQL-NS\Samples\EngineHost\bin\Debug directory. The EngineHost usage syntax is

[View full width]

EngineHost -instanceName <InstanceName> -authenticationMode <Mode> [-sqlUserName <SqlUser>
   -sqlPassword <SqlPassword>]

The argument value placeholders (shown between angle brackets) should be replaced as follows:

  • <InstanceName> specifies the name of the SQL-NS instance for which the engine should be hosted.

  • <Mode> specifies the authentication mode the hosted engine should use to connect to the SQL Server. Allowed values are the string "Windows" (to specify Windows Authentication) or the string "SQL" (to specify SQL Server Authentication).

  • <SqlUser> specifies the name of the SQL user account with which the engine should connect to the SQL Server if SQL Server Authentication is specified. If Windows Authentication is specified, omit the -sqlUserName argument.

  • <SqlPassword> specifies the password of the SQL user account with which the engine should connect to the SQL Server if SQL Server Authentication is specified. If Windows Authentication is specified, omit the -sqlPassword argument.

Invoke EngineHost now, specifying "StockBrokerNMO" as the instance name, the appropriate authentication mode for your system, and, if you're using SQL Server Authentication, the SQL username and password of your test account. If you're using Windows Authentication, the command-line syntax is

[View full width]

C:\SQL-NS\Samples\EngineHost\bin\Debug\EngineHost.exe -instanceName StockBrokerNMO
   -authenticationMode Windows

If you're using SQL Server Authentication and, for example, your SQL test account username and password are "SQL-NS_Test" and "testPassword", respectively, the commandline syntax is

[View full width]

C:\SQL-NS\Samples\EngineHost\bin\Debug\EngineHost.exe -instanceName StockBrokerNMO
   -authenticationMode SQL -sqlUserName "SQL-NS_Test" -sqlPassword "testPassword"

When EngineHost is invoked, it displays a message saying the engine has been started; it then continues running until you press a key. With EngineHost running, you can proceed to exercise the functionality of the StockBrokerNMO instance in the same way you would if the engine had been running in the SQL-NS Windows service.

To see the engine at work, go back to the instructions in the section "Testing the StockBrokerNMO Sample Application" (p. 554) in Chapter 16, and complete step 8. This enters a few sample subscriber and subscription records and then submits a batch of test events. Within 30 seconds, you'll notice new notifications written to the same output file as before, C:\SQL-NS\Samples\StockBrokerNMO\FileNotifications.txt. As this shows, the SQL-NS engine, hosted in the EngineHost application, behaves exactly as it does when hosted in the SQL-NS Windows service.


Summary

The SQL-NS Hosting API can be used to build custom applications that host the SQL-NS execution engine. When hosted in a custom application, the engine behaves the same way it does when running in the SQL-NS Windows service. Hosting the engine provides an opportunity to consolidate the running components of a SQL-NS instance into an existing application process.

The Hosting API consists of members of the NSInstance class. Methods for starting and stopping the engine are provided, along with a property that returns the current state of the engine, and an event used to signal fatal errors. In the hosting application's code, you create an NSInstance object for a particular SQL-NS instance and call the StartInstance() and StopInstance() methods as appropriate. You can also attach an event handler delegate to the Error event, to handle fatal errors in the engine's execution.

This chapter explains the semantics of the hosting API in general and then explores its use in the context of a sample application. This sample application serves as a generic engine host for any SQL-NS instance. Its source code illustrates the essential coding patterns involved in building hosting applications. Instructions in this chapter explain how to use the sample application to host the engine for one of the sample SQL-NS instances developed in a previous chapter.


Chapter 18. User-Defined Matching Logic in SQL-NS Applications

IN THIS CHAPTER

  • Overview of User-Defined Matching Logic

  • Building the Music Store Application with User-Defined Matching Logic

At the center of every SQL-NS application is the core logic that produces notifications by matching events with subscriptions. In all the applications presented so far, this logic is coded in match rulesT-SQL statements that join views of the events and subscriptions and then select only those rows that adhere to some matching criteria. In these match rules, the matching logic is fixed at development time. The end users of the application (its subscribers) can only specify parameters to the developer-defined matching logic; they cannot change the logic itself. Only the developer of the application decides what it means for an event to match a subscription.

SQL-NS offers an alternative model in which end users can specify their own matching logic as part of their subscriptions. This is referred to as user-defined logic. Rather than evaluating some developer-defined match rule, SQL-NS evaluates the custom matching logic in each subscription to determine the matches. In this model, each user can create a unique definition of matching.

This chapter is about building SQL-NS applications that support user-defined logic. It covers the ADF elements used to configure applications for user-defined logic and the API for building subscription management interfaces by which users can specify matching logic when creating subscriptions.


Overview of User-Defined Matching Logic

This section provides an overview of the concepts related to user-defined logic in SQL-NS applications. As you'll see, the user-defined logic model is really an extension of the SQL-NS application model you already know. Applications that use user-defined logic look much the same as those that use developer-defined logicthe few important differences are highlighted here.

Contrasting Developer-Defined Logic and User-Defined Logic

In applications based on developer-defined logic, users are limited to the subscription templates that developers provide in the various subscription classes. To take our music store application as an example, users can subscribe for new song notifications only by artist or by genre, because the logic we implemented matches only on these fields. Users cannot, for example, subscribe by a combination of artist and genre, or by album name, because the application contains no logic that can evaluate such criteria.

User-defined logic lifts this restriction by allowing users to freely construct custom matching logic at subscription time. Returning to the music store example, with user-defined logic, one user could choose to subscribe by song title, another by a combination of album name and genre, and yet another by the number of words in the artist name. Each of these subscriptions would specify its own matching logicthe logic in the first subscription says that a match occurs when the song title in the event matches a particular value, the second says that the album name and genre must equal a pair of specified values, and the third says that the result of a function that counts the words in the artist name must be greater than a specified value.

The important point about user-defined logic is not the sophistication of the custom matching criteria (all the preceding examples could easily have been implemented in developer-defined match rules). Instead, the point is that an almost unlimited variety of subscriptions can be created without a developer first defining templates for their matching logic. The developer just defines a single subscription class and configures it to rely on user-defined matching logic in each subscription. With this model, the developer need not try to anticipate all the kinds of subscriptions that users will want.

The flexibility of the user-defined logic model comes at the expense of scalability. Because each subscription may have unique logic, matching is more expensive than when developer-defined logic is used. With developer-defined logic, all subscriptions of a given subscription class can be evaluated with a single query. User-defined logic may require several queries to evaluate all subscriptions. SQL-NS does perform some optimizations, such as evaluating subscriptions together with structurally similar matching logic, but even so, user-defined logic is generally the less performant of the two models.

Note

For a description of how SQL queries can be used to efficiently evaluate user-defined logic, see the sidebar "Evaluating User-Defined Logic with Queries" (p. 45) in Chapter 3, "The Simplest Notification Application: Stock Quotes."

However, the performance cost of user-defined logic usually isn't prohibitive. Only the very largest applications (those that process millions of events and subscriptions each day) may find performance unacceptable. In most small- to medium-sized applications (and even some fairly large ones), the cost of using user-defined logic is tolerable and a worthwhile trade-off for the flexibility gains.

Note

Empirical evidence indicates that the maximum throughput of an application that uses user-defined logic is about one-fifth that of the equivalent application built with developer-defined logic. Although this fraction may seem alarming, remember that it's onefifth of the incredibly high maximum throughput that SQL-NS normally offers. Few applications actually come close to utilizing the maximum throughput of the SQL-NS engine, so the reduction is less significant than it may seem.

The one-fifth number is a rough guideline, based on a representative set of performance tests. The relative performance of user-defined and developer-defined logic in any given application depends on a number of factors, including the schema of the event data, the average complexity of the subscriptions, and the extent to which subscriptions differ in structure from one another.

Supporting User-Defined Logic in SQL-NS Applications

SQL-NS applications that support user-defined logic are built in essentially the same way as those that support only developer-defined logic. They are defined in ADFs, compiled into database objects, and run by the SQL-NS engine. Their subscription management interfaces (SMIs), built with the SQL-NS Subscription Management API, create and record subscription data in the application database tables.

Support for user-defined logic is specified in an application's subscription classes. Those subscription classes that support custom matching logic define special kinds of event and scheduled rules. Rather than describing developer-defined matching logic, these rules configure the use of user-defined logic: they specify the data over which user-defined logic can be expressed, the security context for the execution of that logic, and the means by which the matches generated by the logic are transformed into notifications. In all other aspects, subscription classes that support user-defined logic are no different from other subscription classes: they can have subscription fields, chronicles, index-creation statements and may specify filegroups for physical storage.

Note

A single application can have some subscription classes that support only developerdefined logic and others that support user-defined logic.

When a user creates a subscription of a subscription class that supports user-defined logic, she must specify the matching logic she wants, in addition to values for any subscription class fields. The matching logic is specified as a Boolean condition over some input data.

You can think of the condition as a logical expression that must evaluate to true for the input data to match the subscription. The input data is usually an event or, in the case of scheduled subscriptions, data stored in a chronicle table. The particular source of input data is defined in the configuration information within the subscription class's rules. The section "Expressing User-Defined Logic with Conditions" (p. 595), explains conditions in more detail and describes the kinds of user-defined logic that can be expressed in the SQL-NS condition model.

The user interface for specifying condition-based matching logic is implemented in the application's SMI. The particular user interface for creating and managing conditions varies from application to application; the SMI designer is free to choose the most appropriate presentation mechanism. But whatever its interface, every SMI uses the SQL-NS Subscription Management API to manage the underlying subscription data. The API provides classes and methods for representing conditions that encapsulate user-defined logic.

In the following section, we look at the ADF elements used to define subscription classes with rules based on user-defined logic. The extensions to the SQL-NS Subscription Management API for user-defined logic are covered in the section "Building Subscription Management Interfaces for User-Defined Logic," (p. 606).

Defining Subscription Classes that Support User-Defined Logic

As you've seen in the sample ADFs in previous chapters, a typical subscription class definition specifies the subscription class name, a set of data fields, event or scheduled rules, and, optionally, chronicles. When you want to support user-defined logic in a subscription class, the only thing that changes is the nature of the rules. Before looking at the ADF elements used to specify user-defined logic, let's briefly recap the basic ideas about rules as we've used them so far (in subscription classes based on developer-defined logic).

The purpose of every rule is to define how notifications are generated from subscriptions. Rules can be of two types: event triggered and scheduled. Event triggered rules are evaluated whenever events of a specified event class (known as the triggering event class) arrive; the logic of an event rule states how events of the triggering event class are matched against subscriptions to form notifications. Scheduled rules are evaluated at the schedule times specified in subscriptions; they describe how scheduled subscriptions are matched, usually against chronicled data, to produce notifications.

A typical event rule declaration has the following form:

<SubscriptionClass>
  ...
  <EventRules>
    <EventRule>
      <RuleName>...Rule Name...</RuleName>
      <Action>
        ...SQL Statement...
      </Action>
      <EventClassName>...Triggering Event Class Name...</EventClassName>
    </EventRule>
  </EventRules>
  ...
</SubscriptionClass>

The three elements within the <EventRule> declaration specify the rule name, an action SQL statement, and the name of the triggering event class.

Scheduled rule declarations look similar:

<SubscriptionClass>
  ...
  <ScheduledRules>
    <ScheduledRule>
      <RuleName>...Rule Name...</RuleName>
      <Action>
        ...SQL Statement...
      </Action>
    </ScheduledRule>
  </ScheduledRules>
  ...
</SubscriptionClass>

Within the <ScheduledRule> element, only a rule name and action are declared. Because scheduled rules are not triggered by events, no triggering event class name is specified.

Note

The <EventRule> element is explained in detail in the section "Declaring Match Rules in the ADF" (p. 155) in Chapter 5, "Designing and Prototyping an Application." The <ScheduledRule> element is covered in the section "Matching Logic for Scheduled Subscriptions" (p. 172), in Chapter 6, "Completing the Application Prototype: Scheduled Subscriptions and Application State."

In both event and scheduled rule declarations, developer-defined matching logic is specified in the <Action> element. The logic takes the form of a SQL statement that expresses the matching criteria.

In this section, we introduce rules that match based on user-defined logic, instead of developer-defined logic. These rules are declared using the same <EventRule> and <ScheduledRule> elements, but the new <ConditionAction> element is used in place of the <Action> element. <ConditionAction> defines the kinds of conditions that subscribers can use to specify custom matching logic and how matches determined by that userdefined logic result in notifications.

The use of <ConditionAction> instead of <Action> is the only structural difference between the declarations of rules based on user-defined logic and those based on developer-defined logic. Conceptually, rules declared with the <ConditionAction> element still define how notifications are generated from subscriptions. They just don't specify the matching logic within the rule definition. Instead, they rely on user-defined logic to do the matching. In the following section, we look in detail at the <ConditionAction> element and its subelements.

The <ConditionAction> Element

The <ConditionAction> element specifies several properties related to the structure and processing of user-defined conditions. Before we look at those properties in detail, it's important to understand where the <ConditionAction> element fits among the other elements in a rule declaration.

The <ConditionAction> element can be used within the <EventRule> element as follows:

<SubscriptionClass>
  ...
  <EventRules>
    <EventRule>
      <RuleName>...Rule Name...</RuleName>
      <ConditionAction>
        ...
      </ConditionAction>
      <EventClassName>...Triggering Event Class Name...</EventClassName>
    </EventRule>
  </EventRules>
  ...
</SubscriptionClass>

As you can see, <ConditionAction> appears in place of the <Action> element. The parts of the event rule declaration that specify the rule name and the triggering event class name remain unchanged.

The <ConditionAction> element can also take the place of the <Action> element within a <ScheduledRule> declaration, to create a schedule rule that performs matching based on user-defined logic:

<SubscriptionClass>
  ...
  <ScheduledRules>
    <ScheduledRule>
      <RuleName>...Rule Name...</RuleName>
      <ConditionAction>
        ...
      </ConditionAction>
    </ScheduledRule>
  </ScheduledRules>
  ...
</SubscriptionClass>

Caution

A single rule may not declare both an <Action> element and a <ConditionAction> element. In other words, you must choose either developer-defined logic or user-defined logic for a particular rule. Within a single subscription class, however, you can declare some rules with <Action> elements and others with <ConditionAction> elements, though this is rarely done in practice.

Whether used in an event rule or a scheduled rule, the <ConditionAction> element has the following internal structure:

<ConditionAction>
  <SqlLogin>...Login Name...</SqlLogin>
  <SqlUser>...User Name...</SqlUser>
  <InputName>...Table or View Name...</InputName>
  <InputSchema>...Schema Name...</InputSchema>
  <SqlExpression>
    ...SQL Statement...
  </SqlExpression>
</ConditionAction>

Within the <ConditionAction> element, the <SqlLogin> and <SqlUser> subelements specify the security context in which user-defined logic is executed when the SQL-NS engine evaluates the rule. These elements and the related security concepts are explained in the section "Securing User-Defined Logic" (p. 593).

The <InputName> and <InputSchema> elements define the input data over which userdefined condition logic can be written. The <InputName> element specifies the name of a table or view in the application's database and the <InputSchema> element specifies the name of the schema containing that table or view.

In event rules, user-defined conditions are usually expressed over the triggering event class's event data. Thus, the <InputName> element usually specifies the name of the triggering event class's event view (which is always the event class name) and the <InputSchema> element specifies the application schema name. In the case of scheduled rules, the input data for user-defined conditions usually comes from a chronicle, so the <InputName> name specifies the chronicle table name and the <InputSchema> element specifies the containing schema.

Note

In general, a rule's input data can come from any table or view in the application database. Although in most cases event rules take their input from event views and scheduled rules from chronicles, this is not a requirement.

The <SqlExpression> element within <ConditionAction> specifies a SQL statement that inserts rows into the notifications view, based on the matches determined by the userdefined logic. To understand the job of this SQL statement, we need to go back to an example of developer-defined logic and take a closer look at the SQL statement from a match rule's <Action> element. Figure 18.1 shows a dissection of the SQL statement taken from the music store application's event triggered match rule.

Figure 18.1. A dissection of the developer-defined SQL statement from a match rule.

[View full size image]

As Figure 18.1 shows, the SQL statement can be divided into three parts. The part shown at the bottom of Figure 18.1 joins the input views based on the matching criteria. The result of these joins is a set of rows that represent the raw matches. The middle part of the statement in Figure 18.1 projects the selected data into a form that matches the schema of the target notification class. The part of the statement shown at the top of the figure inserts the projected data into the notification class's notifications view.

You can think of the three parts of the statement as performing three distinct functions:

1. | Select the matching data.
2. | Project the matching data into a form that corresponds to the schema of the target notification class.
3. | Insert the projected data into the notifications view of the target notification class.

Conceptually, rules based on user-defined logic need to perform these same three functions. But they differ from rules based on developer-defined logic in the mechanism used to select the matching data (the first function in the preceding list).

When evaluating rules based on user-defined logic, SQL-NS engine evaluates each subscription condition against each row in the input table or view. Those subscriptions with conditions that evaluate to true for a given input row are said to match that input row.

Note

For the sake of simplicity, you can imagine the subscription conditions being evaluated one at a time, for each row in the input table or view. However, in practice, SQL-NS uses a more efficient batching strategy to evaluate many conditions at a time. The final results of this evaluation strategy are the same as if the conditions had been evaluated one at a time.

SQL-NS makes the results of the condition evaluation available in a view. This view, referred to as the matches view, contains one row for every match between the subscriptions and the inputs. Each row in the matches view contains all the data from a pair of matching input and subscription rows. Thus, the schema of the matches view is a combination of the schemas of the input table or view and the subscription class view. For matches based on user-defined logic, the data in the matches view roughly corresponds to the results of the joins shown at the bottom of Figure 18.1; it represents the raw matches between the subscriptions and the inputs. The matches view is illustrated in Figure 18.2.

Figure 18.2. The matches view is populated with the results of subscription condition evaluation.

[View full size image]

Data in the Matches View

To understand the contents of the matches view, the following definition may help:

Consider each row in the input table or view to be a tuple, I, of the following form:

  I = (i1, i2, ..., iN)

The values i1, i2,..., and iN correspond to columns in the input table or view.

Treat each subscription as a tuple, S:

  S = (s1, s2, ..., sN)

The values, s1, s2,..., and sN correspond to columns in the subscription class view (derived from the fields in the subscription class schema).

Each subscription also has an associated user-defined condition, C. This can be thought of as a Boolean function that operates over input tuples:

  C(I)    {true | false}

For any given input tuple, I, C evaluates to either true or false.

At the time of rule evaluation, for every combination of I and S, the SQL-NS engine evaluates C. SQL-NS populates the matches view with a tuple, (i1, i2,..., iN, s1, s2,..., sN), if and only if C is true when evaluated against I.

Columns in the matches view that are taken from the input are prefixed with "Input." and those taken from the subscriptions view are prefixed "Subscription.". For example, if the input is a table with columns, A, B, and C, and the subscription class has fields, F1, F2, and F3, the matches view contains the columns Input.A, Input.B, Input.C, Subscription.F1, Subscription.F2, and Subscription.F3. This naming scheme prevents column name collisions and clearly indicates the source of the data in each column of the matches view.

Note

Don't be confused by the use of the period (.) in the names of the matches view's columns. SQL Server considers the period a valid character in column names. The period does not have any special meaning, as it does in languages such as C#when used in a column name, it's just another character.

The SQL-NS compiler creates a dedicated matches view in the application schema for each rule that has a <ConditionAction> element. The name of the matches view is always the rule name.

By populating the matches view, SQL-NS takes care of selecting the matching data (the function fulfilled by the joins shown at the bottom of Figure 18.1) for rules based on userdefined logic. However, the other functions depicted in the figure, projecting the matching data into the form of the notification class schema and inserting it into the notifications view, still need to be performed. This is the job of the SQL expression defined in the <SqlExpression> subelement in the <ConditionAction> element. The SQL expression selects data from the matches view, projecting the appropriate columns, and inserts it into the notifications view. SQL expressions in <ConditionAction> elements generally have the following form:

INSERT INTO [<SchemaName>].[<NotificationClassName>]
SELECT  [Subscription.SubscriberId],
        [...device name...],
        [...locale...],
        [Subscription.<Field1>],
        [Subscription.<Field2>],
        ...
        [Subscription.<FieldN>],
        [Input.<Field1>],
        [Input.<Field2>],
        ...
        [Input.<FieldN>],
FROM    [<SchemaName>].[<RuleName>]

The SQL expression selects from the rule's matches view (which is named the same as the rule) and inserts into a notifications view (named the same as the corresponding notification class). Recall that all notification views have SubscriberId, DeviceName, and SubscriberLocale as their first three columns, so the first three items in the select list provide these values.

Notice that the subscriber ID is taken from the Subscription.SubscriberId column in the matches view. Because every subscription class view always has a SubscriberId column, every matches view always has a Subscription.SubscriberId column.

The device name and locale can come from various sources, as you've seen in previous chapter's examples. In the simplest of applications, the values for these columns are hardcoded. In other applications, they may come from subscription class fields, in which case their values could be selected from the corresponding "Subscription." columns in the matches view.

The remaining columns in the select list correspond to the remaining fields in the notification class schema. Values can be taken from fields in the matches view, given as constants, or computed by SQL expressions.

Note

The SQL expression in the <ConditionAction> element can perform further filtering of the rows in the matches view if additional, developer-defined matching criteria need to be applied, beyond the user-defined matching logic. Such filtering could be implemented in a WHERE clause on the SELECT statement in the SQL expression.

Together, the user-defined subscription conditions, the SQL-NS engine components that populate the matches view, and the SQL statement specified in the <ConditionAction> element perform all the processing necessary to generate notifications. This is equivalent to the processing normally done by the SQL statement in a developer-defined rule's <Action> element.

Processing of Rules with <ConditionAction> Elements

The introduction of user-defined logic into the SQL-NS application model sometimes causes confusion about when rules are evaluated and what data is available at evaluation time. It's a common misconception that the processing of rules with user-defined logic is triggered by changes in the data in the input table or view. This is not the case. Another common misconception is that the generator scopes data in the input table or view, much like it does in the event and subscription views. Again, this is not the case.

To understand how rule processing really works, we need to revisit the quantum processing model. Recall that the generator divides time into discrete, fixed-length intervals called quantums. It processes quantums one by one, in time order. During the processing of a quantum, the generator executes the application's event and scheduled rules over the data that is in scope for the quantum.

In the case of developer-defined logic, the SQL statements in the rules are written against views of the event and subscription data. Before executing the rules, the generator populates these views with the events and subscriptions the rules should process during the current quantum. Only the event batches that were committed between the quantum's start and end times are included in the event views. On the subscription side, all event triggered subscriptions are included in the subscription views, but only those scheduled subscriptions due to be evaluated between the quantum's start and end times are included.

Note

For a thorough description of the basic quantum processing model, see the section "Inside the Notification Generator" (p. 395) in Chapter 11, "Debugging Notification Generation."

User-defined logic does not change the fundamental quantum processing model. Within a quantum, the generator populates event and subscription data in the views according to the quantum's start and end times and uses the same algorithm as before to choose event and scheduled rules to be evaluated.

Event rules are selected based on the event classes of the event batches that are in scope during a quantum. Every event rule that lists one of these event classes as its triggering event class is selected. Scheduled rules are selected based on the subscription classes of the scheduled subscriptions that are in scope. All scheduled rules in these subscription classes are selected. The precise ordering of the event and scheduled rule execution depends on the ordering model selected. See the section "Ordering Models for Rule Execution," (p. 401), in Chapter 11, for a detailed explanation of the supported ordering models.

When a rule defined with a <ConditionAction> element is executed, the associated user-defined conditions are evaluated against the data in the input table or view. It's important to understand that at the time of execution, the generator just treats this input table or view as a source of data. It does not specifically scope the data or build views over it in any way. If the input happens to be one of the event views, the data has already been scoped. In some cases (such as the example we look at later in this chapter), the condition input is a view defined as a join between the events view and some other table or view. In these cases, the data in the input view is effectively scoped as a result of the data in the event view being scoped. However, the generator doesn't manipulate the input data directly. In the case that the input is a table, all data in the table is in scope.

The results of condition evaluation are populated in the matches view, as illustrated in Figure 18.2. Thereafter, the SQL expression defined in the rule's <ConditionAction> element is executed. This inserts rows into some notifications view, based on the data in the matches view. This completes the processing of the rule.

To summarize, the generator uses the standard quantum model to scope event and subscription data. Selection of rules to process within a quantum is based purely on the events and subscriptions in scope, not the data in the input tables or views. At evaluation time, the input table or view serves only as a source of data for the user-defined conditions.

Securing User-Defined Logic

As shown in the earlier section "The <ConditionAction> Element" (p. 586), a rule's <ConditionAction> element contains <SqlLogin> and <SqlUser> subelements. These elements are used to specify a SQL login name and username that define the security context in which user-defined logic executes when the rule is evaluated.

Special security mechanisms are needed because user-defined logic is potentially dangerous. In essence, a user-defined subscription condition is really a piece of code: it defines a predicate that will be executed. Supporting user-defined logic means, therefore, that you're providing external, possibly untrusted, users with a way to inject code into your application. When used properly, this provides legitimate users with a flexible means to customize subscription matching. But malicious users might try to inject harmful code via subscription conditions. If they succeed in doing this, they might gain access to privileged information, bring your application down, or induce it to act on their behalf. The support for user-defined logic in SQL-NS was designed with these security concerns in mind and includes several layers of safeguards to prevent subscription conditions from being used in harmful ways.

First, as you'll see in the section "Expressing User-Defined Logic with Conditions" (p. 595) only safe, read-only operations can be used in condition expressions. Also, the APIs and execution mechanisms for user-defined logic treat all user-supplied data values with suspicion. These values are never directly inserted into executable statements. This greatly reduces the possibility of code injection attacks that might transform safe operations into harmful ones. And all these mechanisms are implemented on top of the already stringent role-based security model used throughout SQL-NS.

As a final safeguard, SQL-NS allows you to configure the engine to run user-defined logic in a low-privilege security context. The SQL-NS engine should already be configured to execute all database operations as a low-privileged account, but you can define an even more restricted security context for user-defined logic (one that doesn't even have permissions to access the internal objects needed by the SQL-NS engine). If this mechanism is used properly, then, even if a malicious user was able to circumvent other safeguards and introduce harmful code into the application by means of a subscription condition, that code will be executed in the context of a user account that lacks the privileges to perform harmful operations.

You control the security context for user-defined condition evaluation via the <SqlLogin> and <SqlUser> elements in the <ConditionAction> declaration. The <SqlLogin> element specifies the name of a login account on the SQL Server. The <SqlUser> specifies a user account in the application's database.

At compile time, the login name specified in the <SqlLogin> element must already exist on the server. In contrast, the username specified in the <SqlUser> element must not exist in the application's database. The SQL-NS compiler creates the user account and associates it with the given login. At runtime, the SQL-NS engine impersonates the specified user before evaluating condition logic. All user-defined logic is then executed in the security context of this user.

Note

Each rule can have its own security context for user-defined logic. The <SqlLogin> and <SqlUser> elements are defined within the <ConditionAction> element, on a per-rule basis. These elements can have different values in different rules.

After compiling the application, you will need to grant any required permissions to the user account you specified in the <SqlUser> element (although the SQL-NS compiler creates the account, it does not grant it any permissions). Typically, very few permissions are needed. In fact, this account usually needs only SELECT permissions on the rule's input table or view. By keeping the account's privileges to a minimum, you can effectively create a sandbox around the user-defined logic, preventing it from executing any harmful operations.

Note

This is a somewhat unusual use of impersonation. Typically, server applications impersonate specific users to temporarily elevate their privileges to perform user-specific operations. Here, impersonation is used to lower the privileges of the SQL-NS engine before executing user-defined logic.

Caution

It is your responsibility to ensure that the login account you specify in the <SqlLogin> element has only the minimum required permissions. Through impersonation, SQL-NS will execute user-defined logic in the context of whatever account you specify in this element. It does not in any way validate that this account is low privileged. If you specify a login with elevated privileges (for example, an account that is a system administrator or has been granted other system permissions), user-defined logic will run with all the corresponding permissions.

Expressing User-Defined Logic with Conditions

SQL-NS applications that support user-defined logic have one or more rules defined with the <ConditionAction> element. When users create subscriptions of the associated subscription classes, they specify their matching logic in terms of conditions on the input data specified in the <ConditionAction> element's <InputName> and <InputSchema> subelements. SQL-NS defines a particular model for expressing conditions. In this section, we take a detailed look at the kinds of matching logic that can be expressed with the SQL-NS condition model.

The SQL-NS Condition Model

As you saw in the applications we looked at in previous chapters, developer-defined matching logic is expressed in the form of T-SQL statements. This is an acceptable way for developers to define matching logic, but it's not reasonable to expect end users to write T-SQL statements to express the custom matching logic they want. Instead, SQL-NS provides a simplified, yet powerful, model for expressing user-defined matching logic in terms that users can understand. As the previous sections of this chapter have mentioned, this model is based on conditions.

When creating a subscription, users are given a description of the input data in the table or view specified in a rule's <ConditionAction> element. They are then asked to specify a Boolean condition expression that, when evaluated against a row of the input data, determines whether a match has occurred. This condition is stored as part of the subscription, along with the regular subscription data (like the values of the subscription fields).

At runtime, the condition is evaluated against the rows in the input table or view. If a subscription's condition evaluates to true for a particular input row, a match has occurred between the subscription and that input row. A new row will be populated in the matches view, containing the data in the input row and the subscription.

As a basis for some examples, imagine that we extended our music store sample application to support user-defined logic. In the ADF, we would declare a subscription class with one event rule, defined with a <ConditionAction> element. The rule's <ConditionAction> element declares its input to be a view over all the properties of the songs submitted as song added events. The view therefore has columns SongId, SongTitle, ArtistName, AlbumTitle, Genre, and DateAdded. (Ignore, for the moment, the question of how such a view might be defined or populated.) Subscriptions of this subscription class express userdefined logic in terms of conditions over this input view.

To create a subscription that matches song added events when the artist name is Miles Davis and the album title is "Kind of Blue," a subscriber could define a condition as follows:

ArtistName = "Miles Davis" AND AlbumTitle = "Kind of Blue"

A subscription that matches song events only when the genre is not "Rock" or "Hip Hop" could express its matching criteria with the condition:

NOT(Genre = "Rock" OR Genre = "Hip Hop")

Although we can't expect users to write T-SQL statements to express their matching logic, we do ask them to specify conditions such as these. The difference may not be immediately apparent, especially because conditions look a lot like the WHERE clauses in T-SQL queries. But writing conditions is fundamentally simpler than writing T-SQL queries because conditions are always defined over a single row of input data. Writing a query, in contrast, requires reasoning about sets of data, and most users don't think this way. Reducing the definition of matching logic to condition expressions over single inputs makes the model accessible to a large base of users. Furthermore, users rarely actually write condition expressions in text (like the examples shown in this section). Instead, they usually construct the conditions using a rich user interface that simplifies the process.

Note

Other systems that ask users to define conditions over single inputs include search engines and rule builders (like those found in many rich email clients). The popularity of these systems suggests that users generally are capable of expressing meaningful logic through conditions.

The SQL-NS condition model consists of a few basic constructs that can be composed to express complex conditions. This section showed a few simple examples of conditions that can be constructed with this model. The following section explains all the supported condition constructs and the ways they can be composed.

Types of Conditions

As you saw from the previous section's examples, conditions are just Boolean predicates. These predicates are constructed from terms, relational operators, and logical operators. This section outlines the types of allowed terms and operators, as well as the ways they can be combined to form complex conditions.

To begin, let's consider the terms (or values) in condition expressions. Terms can be one of the following:

  • A constant value, such as the number 5 or the string "Miles Davis".

  • A field in the input data, such as the ArtistName column in a view of song properties.

  • An arithmetic expression, such as 5 + 2, involving two other terms and the binary arithmetic operators (addition, subtraction, multiplication, and division).

  • The result of a call to a scalar-valued SQL function using other terms as values for the function's arguments; for example, if NumberOfWords() is a SQL function that returns the number of words in a string argument, NumberOfWords(ArtistName) can be a term.

  • A computed property of the input data, as determined by a user-defined property rulefor now, don't be concerned with this type of term; property rules are explained in the section "Using User-Defined Rules to Express Subscription Conditions," (p. 598).

Terms can be composed recursively in arithmetic expressions and function calls. In other words, the individual terms in an arithmetic expression term can themselves be arithmetic expression terms involving simpler terms. For example, the term 5 + (52) is an arithmetic expression term involving two terms: the constant value, 5, and the simpler arithmetic expression term, 52. Similarly, in a functional call term, the terms used as values for the function's arguments may be arithmetic expressions or the results of other function calls.

The SQL-NS condition model defines a set of leaf conditions that can be expressed using the terms in the preceding list. Think of these leaf conditions as templates for the fundamental predicates that can appear in condition expressions. Leaf conditions are the simplest kinds of conditions (so named because they're always expressed in terms of the leaves of a condition expression tree). The supported leaf conditions are

  • Simple leaf conditions that compare two terms with the relational operators =, >, >=, <, <=, LIKE, and != (not equals); for example, ArtistName = "Miles Davis".

  • Is-null leaf conditions that test whether a term is null; for example, if Genre is a nullable field in the input view, IsNull(Genre) returns true if the value of the Genre field is null in the input row.

  • Boolean leaf conditions that test whether a Boolean term's value is true; for example, if the input contains the Boolean field IsHitSong (defined to be of the SQL bit data type), the value of this field can be used as a Boolean leaf condition.

  • Between leaf conditions that determine whether a term's value lies between the values of two other terms; for example, BETWEEN(DateAdded, "1/1/1980", "12/31/1989") is true if the value of the DateAdded field is between January 1, 1980 and December 31, 1989.

  • Link leaf conditions that determine whether rows in another table, related to the input table or view in some way, satisfy the criteria defined by a user-defined filter rule. Ignore this type of condition for now; filter rules and link leaf conditions are explained in the section "Using User-Defined Rules to Express Subscription Conditions," (p. 598).

Any of the term types can be used in simple and between leaf conditions. For example, a simple leaf condition may compare an arithmetic expression term with a constant value term. Is-null leaf conditions are generally used only with terms that are nullable fields in the input data. Boolean leaf conditions can be used with any term that evaluates to a Boolean value.

A subscription condition can consist of a single leaf condition or a combination of several conditions using the logical operators AND, OR, and NOT. The SQL-NS condition model defines the following compound conditions that combine other conditions:

  • AND conditions compute the logical AND of the results of several other conditions.

  • OR conditions compute the logical OR of the results of several other conditions.

  • NOT conditions negate the result of another condition.

As you might expect, compound conditions are also composable recursively. In other words, compound conditions can be formed by logically combining the results of leaf conditions or other compound conditions. For example, consider the following compound condition:

(ArtistName = "Miles Davis") OR
((Genre = "Jazz") AND
 (BETWEEN(DateAdded, "1/1/1980", "12/31/1989")))

This condition contains three leaf conditions: ArtistName = "Miles Davis", Genre = "Jazz", and BETWEEN(DateAdded, "1/1/1980", "12/31/1989"). It has two compound conditions: one OR condition and one AND condition. The OR condition is defined in terms of the first leaf condition and the compound AND condition. The AND condition is defined in terms of a simple leaf condition and a between leaf condition.

Because the SQL-NS condition model does not limit the extent to which terms and conditions can be composed recursively, some very complex conditions can be formed from the few rather simple primitive constructs. As described in the section "Building Subscription Management Interfaces for User-Defined Logic" (p. 606), each of the primitive constructs is represented by a class in the extended SQL-NS Subscription Management API. When building an SMI that allows users to create subscriptions with conditions, you use this API to represent the condition definitions.

Using User-Defined Rules to Express Subscription Conditions

As alluded to in the previous section, some types of terms and conditions make reference to user-defined rules. These user-defined rules are simply reusable pieces of user-defined logic that can be applied across several subscription conditions. Think of a user-defined rule as a part of the logic in a condition, broken out into a separate, reusable piece.

Note

Although the same term, "rule", is used, do not confuse user-defined rules with rules defined in the ADF. A user-defined rule is a piece of reusable user-defined logic that is referenced in subscription conditions. A rule in the ADF defines how notifications are generated from subscriptions, either via developer-defined or user-defined logic.

The SQL-NS condition model supports two kinds of user-defined rules:

  • Property rules that define computed properties of a condition's input data.

  • Filter rules that can be used with link leaf conditions to determine whether the rows in a table or view related to a condition's input (via foreign keys) satisfy some criteria.

These rule types are discussed in detail in the following sections. Note that the topic of user-defined rules is considered an advanced concept within the area of user-defined logic. If the material in this section appears somewhat confusing, don't be concerned. When we look at some concrete examples in a later section, "Creating User-Defined Rules to Build Conditions," (p. 634) the concepts presented here will become easier to understand.

Property Rules

A property rule is a unit of user-defined logic that computes a single, scalar value, based on a row of input data. Property rules are so named because their results can be thought of as computed properties of the input row. In fact, the SQL-NS condition model allows a property rule's result value to be used as a term in a condition expression, as though it was a field of the input data.

Property rules are associated with an input type, which can be any table or view. A property rule definition consists of an ordered list of statements of the following form:

IF <Condition> THEN <Value>

The <Condition> part is a condition over the rule's input data. It is defined using the same constructs as subscription conditions. For example, the condition might be a compound condition that ANDs together two simple leaf conditions that compare fields in the input. The <Value> part of the property rule statement represents a result value and can be defined using any of the terms types supported for condition expressions. For example, the <Value> could be a constant, a field in the input, or an arithmetic expression.

Note

All the support condition and term types are described in the section "Types of Conditions" (p. 596).

When a property rule is evaluated, each of its statements is evaluated, in order. The first statement in which the <Condition> evaluates to true determines the rule's result. The <Value> defined in this statement becomes the final result value for the rule. If no statement's condition evaluates to true, the property rule's value is null.

Note

You can think of the statements in a property rule as being ordered by priority. Earlier statements have higher priority than later statements. If more than one statement's condition evaluates to true, the result value is taken from the earliest statement.

A property rule can be used as a term in any subscription condition based on the same input type. The term's value is the result of evaluating the property rule against the subscription condition's input row. Because property rules are defined independently, they can be reused across several subscription conditions.

Let's continue with the extended music store application described in the previous section and look at an example of a property rule. In this application, the subscription conditions are defined over an input type that contains the properties of songs submitted as song added events. A user can define a property rule over this input that computes a new song property: a Boolean value that indicates whether a song is suitable for dancing. We call this rule CanDance and define it as follows:

CanDance:
    IF Genre = "Hip Hop" THEN TRUE
    IF Genre = "Rock" THEN FALSE
    IF Genre = "Jazz" THEN FALSE

This rule computes the CanDance property of a song, based on user-defined logic involving the song's genre (one of the fields in the input view). The rule definition consists of three statements, each of which determines a result for the computed property when the genre is a particular value. The rule's logic says that CanDance is true when the genre is "Hip Hop", but false when it is "Rock" or "Jazz".

With the CanDance rule defined, it can be used to write a subscription condition. Suppose that a subscriber wants to be notified whenever new songs she can dance to are added to the music store. She could define a subscription with the following condition:

CanDance = TRUE

The CanDance property can also be used alongside other existing properties of the input to define more complex conditions. If the subscriber wanted to be notified if a song is suitable for dancing or if the song's artist is Miles Davis, she could define her subscription condition as follows:

(canDance = TRUE) OR (ArtistName = "Miles Davis")

In the case of the CanDance property rule example, the rule's statements define Boolean results. This is not a requirementproperty rules can be used to define results of any scalar type. Also, property statements need not have constant results, as in this example. Results can be values obtained from existing fields of the input, arithmetic expressions, or any of the other supported term types. In fact, the result of a property rule statement can be the result of invoking another property rule. This kind of composition is possible because a statement's result is just a term and a property rule can be used anywhere a term is expected.

Property rule composition is also possible in another way: the condition in any property rule statement may be defined using a value computed by another property rule. Again, this is possible because property rules can be used as terms, and the conditions in property rules are formed from terms in the standard ways.

Note

For one property rule to be used in the definition of another property rule (either in a statement's condition or result value), the two property rules must have the same input type because they will always be evaluated against the same input row.

The benefit of property rules may not be clear from the simple example given in this section. After all, the logic defined in the property rule could be easily defined directly in the subscription conditions. However, property rules do offer two unique advantages. The first is reusability: the same property rule can be referenced in different subscriptions. Without property rules, the equivalent logic would need to be duplicated in the definition of each subscription condition. The second advantage is that property rules provide a convenient logical abstraction that makes it easier for users to construct complex conditions. When defining a property rule, users only need to think about the logic related to the property it computes. Thereafter, when thinking about subscription conditions, they can regard the computed property as a given, without worrying about its logical definition every time they use it.

Filter Rules and Link Leaf Conditions

As you've seen in the preceding examples, subscription conditions are defined over rows of data in an input table or view. The columns in the input table or view can be used as terms in condition expressions. However, some situations require that conditions be expressed over data that is related to the input, but not stored in the input table or view. Conditions like these can be expressed using filter rules and link leaf conditions. Link leaf conditions join the data in a subscription's input to the data in another table, and filter rules define the criteria that the rows in that other table must meet.

To set the stage for a practical example, consider a stock notification application, similar in concept to the one we looked at in Chapter 3, but designed to work with user-defined logic. Events in the application are stock price changes, each consisting of a stock symbol and a stock price. Imagine that the application has a subscription class with a rule based on user-defined logic. The rule's <ConditionAction> element specifies that the input data for subscription conditions is the stock price change event view. Given this context, it's easy to imagine various subscription conditions that could be written in terms of the stock symbol and stock price in the input data.

But suppose the application also keeps other information about stocks in separate tables. One of these tables stores information about the companies identified by the stock symbols, including the company names and addresses. This table has columns for company name, stock symbol, and company address. A relationship exists between the data in this table and the data in the events view: the two could be joined on their stock symbol columns in a SQL query. This relationship between the events view and the company information table is illustrated in Figure 18.3.

Figure 18.3. Relationship between the stock price change events view and the company information table.

[View full size image]

By means of filter rules and link leaf conditions, subscribers can actually define subscription conditions over the stock price change events that take into account the related data in the company information table. Suppose a subscriber wants to get notified about all stock price changes for companies located in the states of Washington and Oregon. Because the company location is not stored in the events view (the condition input), this condition can't be written using only the columns in the input.

To implement this condition, the subscriber breaks the logic into two pieces. The first piece defines a filter over the rows in the company information table that selects only those companies located in Washington and Oregon. This filter is expressed as a filter rule. The second piece of the logic defines the subscription condition by stating that a stock price change event is a match if the related row in the company information table satisfies the filter. This is expressed with a link leaf condition.

Filter rules are associated with an input table or view. They consist of a series of user-defined statements that specify the conditions under which rows in the input should be included or excluded from a resultset. Each statement in a filter rule has the following form:

IF <Condition> THEN <Action>

As you probably expect by now, the <Condition> is expressed using any of the supported constructs described in the section "Types of Conditions" (p. 596). Note that this condition is expressed over the input of the filter rule, which may not be the same as the input of the subscription condition in which the filter rule will later be used. In our current example, the input table of the filter rule is the company information table, but the input to the subscription condition is the stock price change events view.

The <Action> specified in each filter rule statement is either include or exclude. The include action means that if a row satisfies the statement's condition, it is included in the filter's resultset. The exclude action means the opposite: if the condition is satisfied by a given row, that row is excluded from the resultset. If a particular row satisfies the conditions in more than one statement and those statements specify conflicting actions, the statement that defines the include action wins. Stated another way, an include action always overrides any exclude actions for the same row.

Note

You can think of a filter rule as a user-defined query. But unlike standard SQL queries in which filtering logic must be expressed in a single, possibly complex, predicate, filter rules use a series of simple filter statements to specify which rows to include or exclude from the resultset. This approach tends to be easier for most end users to understand.

Returning to our stock application example, the filter rule that selects companies in Washington and Oregon can be stated as follows:

CompaniesInWashingtonAndOregon:
    IF State = WA THEN Include
    IF State = OR THEN Include

With the filter rule defined, we can write the link leaf condition that implements the matching criteria for the subscription (match stock price changes for companies located in the states of Washington and Oregon). A link leaf condition joins its input row with a target table or view and then checks whether the related rows in that target table or view appear in the resultset of a filter rule. The target table or view is the input to the filter rule.

The definition of a link leaf condition has the following elements:

  • A link operator

  • A filter rule

  • A series of join clauses that relate the condition input to the input of the filter rule

The link operator defines how many of the related rows in the target table or view must be in resultset of the filter rule to satisfy the condition. Two link operators are supported: ANY and EVERY. If the ANY operator is used, the link leaf condition is satisfied if any of the rows in the target table or view that relate to the input row are in the resultset of the filter rule. As its name suggests, the EVERY operator requires that every row in the target table or view that is related to the input be in the resultset of the filter rule. When the condition input and the target table or view have a one-to-one relationship, the ANY and EVERY operators are equivalent. However, if the relationship is one-to-many, the distinction between the two operators is meaningful.

Note

The two link operators give different results in the case in which no rows in the target table or view are related to the condition's input row. In this case, link leaf conditions that specify the EVERY operator are trivially satisfied. However, link leaf conditions that specify the ANY operator are not satisfied.

The join clauses specified in the link leaf condition define exactly how the input row relates to the table or view over which the filter rule is defined. To understand the content of the link leaf condition join clauses, imagine you had to write a SQL query that joined the condition's input table or view with the filter rule's input table or view. Continuing the example of our stock application, the query that joins the stock price change events view (the input to the subscription conditions) and the company information table (the input to the filter rule) might be written as follows:

SELECT ...
FROM  StockPriceChange
    JOIN CompanyInformation
        ON StockPriceChange.StockSymbol = CompanyInformation.StockSymbol

The JOIN clause in this query specifies the columns that relate the StockPriceChange view to the CompanyInformation table. Join clauses in link leaf conditions are similar, but rather than specifying actual join predicates in T-SQL syntax, they specify pairs of columns, one in each of the tables or views to be joined. It is assumed that these columns join the two tables or views via equality predicates. In the preceding example, the join clause can be expressed as the pair of columns (StockPriceChange.StockSymbol, CompanyInformation.StockSymbol).

With this information, the complete link leaf condition that defines the subscription's match criteria can be written as follows:

LinkLeafCondition(
    LinkOperator: Any,
    FilterRule: CompaniesInWashingtonAndOregon
    JoinClauses:
        (StockPriceChange.StockSymbol, CompanyInformation.StockSymbol)
)

The link leaf condition specifies the link operator ANY, the filter rule we defined earlier, CompaniesInWashingtonAndOregon, and a join clause that specifies the columns used to relate the condition's input view with the filter rule's input table. When this condition is evaluated against an input event row, the join clauses are used to determine the related rows in the CompanyInformation table, the filter rule's input. These rows are then checked against the rows that result from evaluating the filter rule. If any row related to the input is in the filter rule's resultset, the condition is satisfied. Because the filter rule selects only companies in Washington and Oregon, the condition is satisfied only for event rows in which the stock symbol refers to a company in Washington or Oregon. This fulfills the original intent of the subscription.

In this example, the input view can be joined directly to the target table. In other situations, the join requires one or more intermediate tables. For example, imagine that the company information table was split into two tables: a CompanyStockSymbols table that just stores the stock symbols for each company and a CompanyDetails table that stores the rest of the information (the company name and address details). The columns in the CompanyStockSymbols table are CompanyID and StockSymbol; the columns in the CompanyDetails table are CompanyID, CompanyName, Address, City, State, and Zip.

Given this schema, the filter rule that selects companies in Washington and Oregon would need to be defined over the CompanyDetails table (because this is where the address information is stored). However, this table cannot be joined directly to the stock price change events view because no common column exists: the CompanyDetails table does not store stock symbols and the stock price change events do not include company IDs. In this case, the join needs to involve the intermediate table, CompanyStockSymbols. The stock price change events view can be joined to CompanyStockSymbols on stock symbol, which can then be joined to CompanyDetails on company ID. In this case, a set of two join clauses is needed: (StockPriceChange.StockSymbol, CompanyStockSymbols.StockSymbol) and (CompanyStockSymbols.CompanyID, CompanyDetails.CompanyID). You can think of these join clauses as forming a chain between the input view and the target table. This is illustrated in Figure 18.4.

Figure 18.4. Using an intermediate table to join the condition input and the input of the associated filter rule.

[View full size image]

Tip

If a link leaf condition and its associated filter rule are defined against the same input table or view, no join clauses are needed.

Link leaf conditions support arbitrarily long chains of join clauses. The first column in the first join clause must be a column in the condition's input and the second column in the last join clause must be a column in the filter rule's input. In each join clause in between, the first column must be in the same table or view as the second column of the previous join clause.

Note

SQL-NS does not explicitly define a limit on the number of join clauses in a link leaf condition. However, in practice, the length will be limited by the maximum query length supported by SQL Server. This should not be a concerncases that require more than four or five join clauses (well within the number supported) are exceedingly rare.

In the section "Creating Filter Rules and Link Leaf Conditions" (p. 638), we look at the code required to create a filter rule and link leaf condition in the context of a real SQL-NS application. The example presented in that section illustrates the concepts introduced here.

Building Subscription Management Interfaces for User-Defined Logic

The SMIs to SQL-NS applications that support user-defined logic must provide users with a way to construct conditions at the time they create subscriptions. The user interface for building condition expressions depends on the particular application. Some applications provide something like a query-builder interface; others present user-interface controls that allow users to drag and drop elements into condition templates. Some advanced applications even allow users to specify condition logic using a textual syntax.

Whatever the form of the user interface, SMIs that support user-defined logic use the SQL-NS Subscription Management API to create and manage subscription records that include user-defined conditions. In Chapter 7, "The SQL-NS Subscription Management API," we looked at the basic API used to manage subscriptions in applications based on developer-defined logic. In this chapter, we examine the extensions to that API for user-defined logic.

This section provides an overview of the API and the patterns used to program with it. In the later section "Building the Music Store Application with User-Defined Matching Logic," we look at practical code examples based on the API described here.

Note

As usual, the coverage in this book is not intended to be a complete class reference. Reference documentation for all the classes, methods, and properties described here is available in the SQL-NS Books Online.

The Condition API

The API for constructing subscriptions with user-defined conditions is an extension of the SQL-NS Subscription Management API covered in Chapter 7. When building an SMI for an application based on user-defined logic, the same basic classes in that API, Subscriber, SubscriberDevice, and Subscription, are used to represent subscribers, subscriptions, and subscriber devices. The new classes and methods introduced in this section are used to represent user-defined conditions.

When creating subscriptions, the SMI code creates Subscription objects in the usual way, initializes their basic properties, and sets their subscription fields. When these subscriptions are of subscription classes with rules based on user-defined logic, two new properties must also be set on the Subscription objects: RuleName and Condition. The RuleName property is set to the name of a rule defined in the subscription class with a <ConditionAction> element. The Condition property is set to a condition object that represents the user's subscription condition. When the Add() method is called on a Subscription object that has these properties set, the condition logic specified in the condition object is validated and then stored in the database, along with the rest of the subscription information.

Condition objects are constructed from various classes that the API provides to represent the elements of a user-defined condition. The SMI code creates instances of these classes and assembles them into condition structures, based on the choices the user makes in the SMI's user interface. The design of the user interface controls and the techniques for gathering information from them are outside the scope of this chapter. Here we focus on the specific classes in the API used to represent conditions. These can be used with any user-interface implementation.

The type of the Subscription class's Condition property is the class Condition. Condition is an abstract base class from which several concrete condition classes are derived. Each of these condition classes represents one of the types of conditions described in the section "Types of Conditions," (p. 596). Specifically, the API defines the following condition classes to represent the various leaf conditions:

  • SimpleLeafCondition represents simple leaf conditions.

  • IsNullLeafCondition represents is-null leaf conditions.

  • BooleanLeafCondition represents Boolean leaf conditions.

  • BetweenLeafCondition represents between leaf conditions.

  • LinkLeafCondition represents link leaf conditions.

To construct a simple leaf condition, for example, the SMI code creates an instance of the SimpleLeafCondition class and sets its properties to specify the various subelements of a simple leaf condition expression.

Compound conditions are represented by the classes AndCondition, OrCondition, and NotCondition. Instances of AndCondition and OrCondition are initialized with lists of other condition objects that represent the subconditions to be combined with the AND or OR operators. NotCondition objects are initialized with a single other condition object representing the condition to be negated.

Terms in condition expressions are represented by value objects. Value objects are instances of classes that inherit from the abstract base class Argument. The API provides a concrete class (derived from Argument) for each supported term type described in the earlier section "Types of Conditions" (p. 596). Specifically, the following value classes are available:

  • ConstantValue represents terms that are constants.

  • FieldValue represents terms that are fields of the input data.

  • ExpressionValue represents terms that are arithmetic expressions.

  • FunctionValue represents terms whose values are obtained by calling SQL functions.

  • RuleValue represents terms whose values are defined by user-defined property rules.

Objects of these classes are initialized with values specific to the types of terms they represent. For example, ConstantValue objects are initialized with the literal values they express. FieldValue objects are initialized with the names of fields in the input data.

Value objects are used with condition objects to fully specify condition expressions. For example, a simple leaf condition that represents an equality comparison between a field in the input and a constant value is manifested as a SimpleLeafCondition object, initialized with a FieldValue object, the equality operator, and a ConstantValue object.

The operators that apply to the various condition types are represented by enumerations. For example, the relational operators supported by simple leaf conditions are represented by members of the SimpleOperator enumeration. Similarly, the link operators used in link leaf conditions are represented by members of the LinkLeafOperator enumeration. When initializing SimpleLeafCondition or LinkLeafCondition objects, the values in the corresponding enumerations are used to specify the operators.

The two kinds of user-defined rules in the SQL-NS condition model are represented by the classes PropertyRule and FilterRule, respectively. These classes encapsulate the definitions of each type of rule. PropertyRule objects are initialized with a set of PropertyStatement objects, each representing one of the statements that make up a property rule definition. Similarly, FilterRule objects are initialized with FilterStatement objects that represent filter rule statements.

As described in the section "Using User-Defined Rules to Express Subscription Conditions" (p. 598), both property rule statements and filter rule statements contain conditions. As you would expect, these are represented programmatically with the same condition classes used to represent subscription conditions. Both the PropertyStatement and FilterStatement classes have Condition properties of the base Condition type that can be set to instances of any of the concrete condition classes described in this section.

Because each property statement in a property rule defines a result value, the PropertyStatement class has a Result property. The type of this property is the abstract base class Argument, from which all the value classes derive. Thus, the Result property can be set to an object of any of the value classes.

Filter statements each specify an include or exclude action. On the FilterStatement class, this is represented by the Action property. The type of this property is the FilterAction enumeration, which has the members Include and Exclude. When creating a FilterStatement object, its Action property can be set to one of these two values.

The classes that make up the Condition API are defined in a separate namespace and implemented in a separate assembly from the rest of the SQL-NS API. All the condition, value, rule, and rule statement classes described in this section (as well as the associated enumerations) are defined in the Microsoft.SqlServer.NotificationServices.Rules namespace. Code files that use the Condition API classes must either declare this namespace in a using directive or use the namespace to fully qualify the class names. The implementation of the Condition API classes is packaged into the Microsoft.SqlServer.NotificationServices.Rules.dll assembly. Projects that use the Condition API must reference this assembly.

Because some of the classes in the standard SQL-NS Subscription Management API, such as Subscription, Subscriber, and SubscriberDevice, are also needed in most applications that use the Condition API, the main SQL-NS namespace, Microsoft.SqlServer.NotificationServices, is generally referenced alongside the Microsoft.SqlServer.NotificationServices.Rules namespace. Accordingly, the main SQL-NS assembly, Microsoft.SqlServer.NotificationServices.dll is also usually referenced in addition to the assembly containing the Condition API.

In the section "The AddSubscriptions Program" (p. 617), later in this chapter, we look at some code samples that make use of the classes described in this section. Those samples illustrate the common patterns in which these classes are used.

Composition of Classes in the Condition API

The "Types of Conditions" section (p. 596) explains how various elements of condition expressions can be composed to form more complex expressions. This composition capability is reflected in the design of the Condition API classes.

Several of the condition classes have properties that refer to other condition objects. For example, the AndCondition class is initialized with a list of condition objects representing conditions to be combined with the AND operator. In the Condition API, references to condition objects are always defined in terms of the Condition abstract base class. In the example of the AndCondition class, the list of child conditions is defined as an array of Condition objects.

Because all the concrete condition classes derive from the common base class Condition, references to Condition can always be fulfilled with an object of any of these concrete classes. Returning to the example of the AndCondition class, the array of child condition objects can be populated with several objects of the various leaf condition classes (SimpleLeafCondition, IsNullLeafCondition, BooleanLeafCondition, BetweenLeafCondition, and LinkLeafCondition). Furthermore, because the compound condition classes are themselves descendants of the base Condition class, instances of these classes can also be used as child condition objects in other compound conditions. So the array of condition objects in an AndCondition object, for example, may include other AndCondition objects, each of which composes several other condition objects. This kind of composition can be repeated to form increasingly complex condition expressions.

Like the condition classes, the value classes also derive from a common base class, Argument. Therefore, the same type of composition based on polymorphism is possible with values. References to value objects in the various methods and properties on the API classes are defined in terms of the base type, Argument. These references can be set to objects of any of the concrete value classes. For example, the ExpressionValue class has two Argument properties that represent the two values to combine in an arithmetic expression. These Argument properties can be set to objects of any of the value classes, including other ExpressionValue objects. Thus, an ExpressionValue object can represent complex expressions consisting of combinations of other expressions, constants, field values, rule values, and function values. This ExpressionValue object can then be used in condition objects, in any place a value object is needed.

Caution

Composition introduces the possibility of cycles in a condition or term definition. For example, a condition object may reference a property rule object which references the same condition object. When constructing condition objects, you should be careful not to introduce cycles of this kind. SQL-NS checks condition definitions for cycles when the associated subscriptions are added to the database. If a cycle is detected, adding the subscription fails.


Building the Music Store Application with User-Defined Matching Logic

In this section, we look at a working example of a SQL-NS application based on user-defined logic. The application is a modified version of the music store sample we developed in previous chapters. It provides the same type of functionalitynotifications about new songs added to the music store catalogbut allows its subscribers to define their own matching criteria. Unlike the previous version of the application, in which the developer-defined matching logic supports only subscriptions based on artist name or genre, the version of the application we build in this chapter lets subscribers create subscriptions with arbitrary matching logic based on song properties.

The complete code for this new version of the music store sample is located in the directory C:\SQL-NS\Samples\UserDefinedLogic. The files in this directory are organized into a similar structure as the ones in the previous music store sample. In the sample directory, you'll find an ICF, ADF, and supporting build scripts. The sample directory also contains a Visual Studio solution with the source code for a program that adds subscriptions to the application using the Condition API described in the previous section. In the sections that follow, we take a detailed look at the application's code in the ADF, the program that adds subscriptions, and the scripts used to configure the security context in which the application executes user-defined logic.

Defining the Sample Application

The music store application was first defined in Chapter 5. The version of the application we create in this chapter is simply a modification of this previous version. Much of the code in the ADF is the same across both versions, so if you've read through Chapter 5, you'll already be familiar with much of the code in this chapter's ADF. This section highlights the changes to the ADF that were made to accommodate user-defined logic.

In Chapter 5, the application was built incrementally as each new SQL-NS concept was introduced. Because you're probably familiar with all the basic SQL-NS concepts by now, this chapter's sample is presented in a completed form. The various sections of the ADF code are discussed one at a time in the following sections, but we do not build the application incrementally.

This chapter's sample application is presented in prototype form, at about the same stage of maturity as the prototype at the end of Chapter 5: its basic functionality works, but it isn't connected to real event sources or delivery systems. The techniques covered in Chapters 6 through 10 for advancing a prototype toward production readiness could be applied to the prototype developed in this chapter, as they were to the previous version.

The following are the key differences between this chapter's ADF and the one we had developed by the end of Chapter 5:

  • A new event chronicle has been added to serve as the input to user-defined subscription conditions.

  • The application defines a single subscription class that supports only user-defined matching logic.

  • The match rule in the subscription class is defined with a <ConditionAction> element.

We look at each of these differences in the subsections that follow.

Note

The ICF for this chapter's sample is essentially the same as the one used in Chapter 5. The only difference is in the default value defined for the _InstanceBaseDirectoryPath_ parameter. Because nothing substantive has changed in the ICF, it isn't discussed in detail in this chapter. If you're interested, you can examine the ICF by opening the file C:\SQL-NS\Samples\UserDefinedLogic\InstanceConfiguration.xml. Refer to Chapter 4, "Working with SQL-NS Instances," for detailed information on the ICF elements.

Creating the Input: The SongAddedDetails View

In the music store application, the events represent the addition of new songs to the music store catalog. In the original design, we chose to define the SongAdded event class with a single data field containing the song ID (see the section "Events," p. 135, in Chapter 5 for details). This minimized the duplication of data; because the music store catalog already stored all the song properties, it wasn't necessary to copy them into the event data. In the previous version of the application, we could easily obtain all the song properties by joining the event data with the music store catalog's SongDetails view in the developer-defined match rule.

When we move to user-defined logic, this events design is somewhat more troublesome. Conceptually, events are the inputs to the user-defined subscription conditions. We want our subscribers to be able to construct meaningful conditions over these inputs, but if the input data contains just the song IDs, this isn't really possible. Most interesting matching logic can be expressed only in terms of the other song properties (such as the song title, artist name, album title, and genre).

To deal with this situation, we have three options:

  1. Leave the design as it is and require that all subscription conditions be written as link leaf conditions that join the event data with the SongDetails view and then use user-defined filter rules over SongDetails to express the matching criteria.

  2. Abandon the goal of minimizing data duplication and expand the event class definition to include all the song properties.

  3. Define a new view to serve as the condition input. This view joins the events view with the SongDetails view and thus provides all the song properties of the songs referenced in the event data.

The first option is undesirable because of the complexity it would add to the development of both the application and its SMI. The second option is undesirable because of its negative performance implications. That leaves the third option, which requires a little extra code up front, but then allows the rest of the application development to proceed in a straightforward manner. In this section, we look at the ADF code required to implement this option.

Listing 18.1 shows the definition of a new chronicle, SongAddedDetails, in the SongAdded event class. This chronicle is the view that will be used as the input to our subscription conditions.

Listing 18.1. Definition of the SongAddedDetails View as an Event Chronicle

<Application>
  ...
  <EventClasses>
    <EventClass>
      <EventClassName>SongAdded</EventClassName>
      <Schema>
        <Field>
         <FieldName>SongId</FieldName>
         <FieldType>INT</FieldType>
         <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
      <Chronicles>
        <Chronicle>
          <ChronicleName>SongAddedDetails</ChronicleName>
          <SqlSchema>
            <SqlStatement>
              -- Drop the view if it exists
              IF EXISTS (
                  SELECT so.name
                  FROM   sys.objects so
                  JOIN   sys.schemas sc
                      ON so.schema_id = sc.schema_id
                  WHERE so.name = 'SongAddedDetails'
                      AND so.type = 'U'
                      AND sc.name = 'SongAlerts')
              DROP VIEW [SongAlerts].[SongAddedDetails]
            </SqlStatement>
            <SqlStatement>
              -- Create the view
              CREATE VIEW [SongAlerts].[SongAddedDetails] AS
              SELECT e.SongId AS SongId,
                     sd.SongTitle AS SongTitle,
                     sd.ArtistName AS ArtistName,
                     sd.AlbumTitle AS AlbumTitle,
                     sd.GenreName AS Genre,
                     sd.DateAdded AS DateAdded
              FROM   [SongAlerts].[SongAdded] e
              JOIN   [Catalog].[SongDetails] sd
                  ON e.SongId = sd.SongId
            </SqlStatement>
          </SqlSchema>
        </Chronicle>
      </Chronicles>
    </EventClass>
  </EventClasses>
  ...
</Application>

The chronicle definition consists of two SQL statements. The first one drops the SongAddedDetails view if it already exists. (As in other chronicles you've seen, this handles the case in which updating the instance causes the chronicle statements to be rerun.) The second statement creates the view. Notice that the view definition is a join between the SongAdded events view and the SongDetails view in the music store catalog schema. The columns selected in the view definition include all the song properties.

This isn't really a chronicle in the traditional sense. Usually chronicles are tables and store data maintained by chronicle rules. The chronicle we've created here doesn't have any chronicle rules because it doesn't actually store any data that needs to be maintained. It is just a view over existing data. We've taken advantage of the fact that SQL-NS allows any valid T-SQL statements within a chronicle's definition. We've just used the chronicle as a convenient place to define the view and have the SQL-NS compiler create it at the time the application is compiled.

Because the schema of the new view, SongAddedDetails, is the same as the music store catalog's SongDetails view that existed previously, you might wonder why the new view is needed. The answer is about scoping. The SongDetails view returns data about all songs in the music store catalog. If our subscription conditions used this view as their source of input data, they would always be evaluated against every song in the music store catalog. Clearly, this is the wrong behavior.

The correct behavior is for the subscription conditions to be evaluated against only the song data for the SongAdded events being processed. At runtime, the SQL-NS engine takes care of populating the SongAdded events view with only the event data that each rule evaluation should see. Because the SongAddedDetails view joins with this events view, it always returns song data for only the in-scope events. Using this view as the input for the subscription conditions means that they are naturally evaluated against the correct data.

Defining the Subscription Class for Subscriptions with User-Defined Logic

In Chapter 5's version of the music store application, we defined a subscription class called NewSongByArtist to represent subscriptions by artist name. Later, we added a NewSongByGenre subscription class for subscriptions by genre. In the new version of the application, we don't define the matching criteria at development time so we don't need specialized subscription classes. Instead, we define a single, generic subscription class from which users can create subscriptions by artist, by genre, or by any other matching criteria they choose.

The definition of this subscription class, called NewSongsOfInterest, is shown in Listing 18.2.

Listing 18.2. Definition of the NewSongsOfInterest Subscription Class

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongsOfInterest</SubscriptionClassName>
      <Schema>
        <Field>
          <FieldName>DeviceName</FieldName>
          <FieldType>NVARCHAR(255)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
        <Field>
          <FieldName>Locale</FieldName>
          <FieldType>NVARCHAR(10)</FieldType>
          <FieldTypeMods>NOT NULL</FieldTypeMods>
        </Field>
      </Schema>
      <EventRules>
        <EventRule>
          <RuleName>MatchNewSongsOfInterest</RuleName>
          <ConditionAction>
            ...
          </ConditionAction>
          <EventClassName>SongAdded</EventClassName>
        </EventRule>
      </EventRules>
    </SubscriptionClass>
  </SubscriptionClasses>

  ...
</Application>

The old NewSongByArtist and NewSongByGenre subscription classes defined fields that were used in the developer-defined matching logic (ArtistName and GenreName fields, respectively). These aren't needed in our new subscription class because parameters like these can be defined directly in each user's condition logic. Instead, our subscription class defines two fields that aren't related to matching logic: DeviceName and Locale. These fields specify the name of the target subscriber device and locale for each subscription. In Chapter 5, our music store application didn't have these fields, but we added them in Chapter 9, "Content Formatters," when we added support for device and locale-specific formatting. We could have left these fields out of this chapter's sample because we don't do any device or locale-specific formatting, but they are included simply to illustrate that subscription classes based on user-defined logic can define fields.

Our subscription class declaration contains a single event rule, MatchNewSongsOfInterest, that is defined with a <ConditionAction> element. Notice that the SongAdded event class is specified as the triggering event class for this rule. The rule's <ConditionAction> element is described in detail in the following section.

Implementing the <ConditionAction> Element

Listing 18.3 shows the definition of the <ConditionAction> element of the MatchNewSongsOfInterest event rule.

Listing 18.3. Definition of the <ConditionAction> Element in the MatchNewSongsOfInterest Event Rule

<Application>
  ...
  <SubscriptionClasses>
    <SubscriptionClass>
      <SubscriptionClassName>NewSongsOfInterest</SubscriptionClassName>
      ...
      <EventRules>
        <EventRule>
          <RuleName>MatchNewSongsOfInterest</RuleName>
          <ConditionAction>
            <SqlLogin>LowPrivEvaluator</SqlLogin>
            <SqlUser>LowPrivEvaluator</SqlUser>
            <InputName>SongAddedDetails</InputName>
            <InputSchema>SongAlerts</InputSchema>
            <SqlExpression>
              INSERT INTO [SongAlerts].[NewSong]
              SELECT  [Subscription.SubscriberId],
                      [Subscription.DeviceName],
                      [Subscription.Locale],
                      [Input.SongTitle],
                      [Input.ArtistName],
                      [Input.AlbumTitle],
                      [Input.Genre]
              FROM    [SongAlerts].[MatchNewSongsOfInterest]
            </SqlExpression>
          </ConditionAction>
          <EventClassName>SongAdded</EventClassName>
        </EventRule>
      </EventRules>
    </SubscriptionClass>
  </SubscriptionClasses>

  ...
</Application>

As described in the earlier section "Securing User-Defined Logic" (p. 593), the <SqlUser> element specifies the user to impersonate when executing user-defined logic for the rule. The <SqlLogin> element specifies the associated server login. In this example, both the login name and username are LowPrivEvaluator. This is the name of a new, low-privilege server login we will create before we compile the application. The SQL-NS compiler will then create a user account of the same name in the application's database. When we grant database permissions to the SQL-NS engine, we'll also grant this user account the minimum permissions it needs to be able to evaluate user-defined logic. The section "Security Configuration for User-Defined Logic," (p. 641) describes the configuration steps needed to create the LowPrivEvaluator login and grant the necessary permissions to the associated user account.

The <InputName> and <InputSchema> elements in the <ConditionAction> declaration specify the name of the input view and the name of its containing schema. The <InputName> element specifies the value SongAddedDetails: the name of the view defined in Listing 18.1. The <InputSchema> element specifies the name of the application schema, SongAlerts, in which the view was created.

Finally, the <ConditionAction>'s <SqlExpression> element defines the SQL statement that inserts rows into the notifications view based on the matched data in the matches view. As described earlier, in the section "The <ConditionAction> Element" (p. 586), the name of the matches view is the rule name. Here, the rule name is MatchNewSongsOfInterest, so the SQL expression selects from a matches view of the same name, in the application schema.

As illustrated in Figure 18.2, the matches view contains a combination of the columns in the input table or view and the fields of the subscription class. Here the input is the SongAddedDetails viewall the columns from this view are included in the matches view, their names prefaced with Input.. Our subscription class defines two fields, DeviceName and Locale, and these are included in the matches view as the columns Subscription.DeviceName and Subscription.Locale.

The SQL expression in the <ConditionAction> element inserts into the NewSong notification class's notifications view, obtaining the subscriber ID, device name, and locale from the subscription columns in the matches view and the rest of the song properties from the input columns. As you can see from this example, the SQL expression is actually quite simple because all the complex logic associated with matching has already been performed. The SQL statement specified here consumes the results of this matching from the matches view.

Note

The NewSong notification class in this chapter's application is the same as the one defined in Chapter 5. You can refer to the section "The Output of the Application: Notifications" (p. 143) in Chapter 5 for a detailed description of this notification class.

The ADF content shown in Listings 18.1, 18.2, and 18.3 is already in the sample's ADF, C:\SQL-NS\Samples\UserDefinedLogic\SongAlerts\ApplicationDefinition.xml. Instructions for compiling and testing this code are provided in the section "Testing the Music Store Application with User-Defined Logic" (p. 643). But before we get there, we first look at the application's subscription management code in the following section.

The AddSubscriptions Program

In this section, we take a detailed look at a program called AddSubscriptions that takes the place of a real SMI for our sample application. AddSubscriptions uses the same SQL-NS Subscription Management API that a real SMI would use (including the user-defined logic extensions described in the section, "The Condition API," p. 607). However, AddSubscriptions lacks a user interfaceit is a command-line program that creates a set of subscriptions based on hard-coded values. These subscriptions illustrate the various aspects of the Condition API and are representative of the kinds of subscriptions users might create. Looking at a simple program like AddSubscriptions, instead of a fully functional SMI, allows us to focus on small code samples specific to the Condition API, rather than the details of a particular user-interface implementation.

Note

When you look at the code for the AddSubscriptions program, you'll see that the condition definitions, though created using the Condition API, are hard-coded into the program. In a real SMI, the condition structures and the values in them would be determined by a user, via the appropriate user-interface controls. The SMI code would translate these into the appropriate Condition API objects.

This sample might give the impression that the Condition API is simply a programmatic way to specify developer-defined logic. This is not the case. The Condition API is intended for use in code that dynamically creates conditions based on user input.

The AddSubscriptions Visual Studio Solution

The code for the AddSubscriptions program is defined in the Visual Studio solution, C:\SQL-NS\Samples\UserDefinedLogic\AddSubscriptions\AddSubscriptions.sln. Open this solution in Visual Studio so that you can refer to the source files as you read through this section.

The source code in the solution is divided into two main files:

  • Program.cs contains the main driver code for the application. In this section, we take a detailed look at the code in this file.

  • Arguments.cs contains code relating to processing of the program's command-line arguments. This code is not relevant to the Condition API or user-defined logic in general, so it isn't covered here.

An outline of the code in Program.cs shown in Listing 18.4.

Listing 18.4. Structure of the Main AddSubscriptions Program Code

...
using Microsoft.SqlServer.NotificationServices;
using Microsoft.SqlServer.NotificationServices.Rules;
using Microsoft.SqlServer.Management.Common;
using Microsoft.SqlServer.Management.Smo;

namespace AddSubscriptions
{
   ...
   class Program
   {
       ...
       private static string sqlServer;
       private static AuthenticationMode authenticationMode;
       private static string sqlUserName;
       private static string sqlPassword;

       static int Main(string[] args)
       {
           ...Argument processing code...
           try
           {
               ...Establish database connections and related objects...

               ...Add subscriptions...
           }
           catch (NSException nsEx)
           {
               ...Handle Notification Services exceptions...
           }
           catch (Exception ex)
           {
               ...Handle general exceptions...
           }

           ...
       }
       ...Helper methods...
   }
}

The using declarations at the top of the file specify both the main notification services namespace and the namespace containing the classes in the Condition API, Microsoft.SqlServer.NotificationServices.Rules. The two other namespaces declared, Microsoft.SqlServer.Management.Common and Microsoft.SqlServer.Management.Smo, contain classes in the SMO library. These namespaces are explained in detail in the section "NMO API Assemblies and Namespaces" (p. 532) in Chapter 16, "Using Notification Services Management Objects." They are needed here because the AddSubscriptions code creates some SMO objects, as described in the section "Obtaining Needed SMO Objects" (p. 622).

All these namespaces are available because the program's solution references the appropriate assemblies. In Visual Studio, if you expand the project's References node in the Solution Explorer tree, you'll see the following assembly references (in addition to the usual system assembly references):

  • Microsoft.SqlServer.ConnectionInfo.dll

  • Microsoft.SqlServer.NotificationServices.dll

  • Microsoft.SqlServer.NotificationServices.Rules.dll

  • Microsoft.SqlServer.Smo.dll

The first and last assemblies in this list contain the needed SMO classes. The other two provide all the SQL-NS-specific APIs. Programs that make use of the Condition API usually require all these assembly references.

Returning to Listing 18.4, notice that the main code for the program is implemented in the Program class. This class declares four private member variables that correspond to the command-line arguments the program expects: the name of the SQL Server containing the music store database, the authentication mode to use when connecting to it, and a SQL username and password to use for SQL Server authentication.

Within the Program class's Main() method (the program's entry point), the first part of the code handles the processing of the command-line arguments. This code parses and validates the arguments and then stores their values in the four private member variables shown in Listing 18.4. The argument processing code is not shown in Listing 18.4 or discussed in this chapter. You can look at this code in the source file if you choose, but for the purposes of this discussion, you can just assume that the values of the command-line arguments end up in the four private member variables shown.

After processing the command-line arguments, the code within the Main() method's try-catch block implements the main functionality of the program. The first part of this code establishes the database connections and other objects needed to construct subscriptions and conditions later. This code is described in the "Connecting to the SongAlerts Application" section (p. 620) and in the "Obtaining Needed SMO Objects" section (p. 622).

The remaining code creates several subscriptions with a variety of conditions. This is the code that illustrates the use of the Condition API. In the sections that follow, we examine this code in detail.

Connecting to the SongAlerts Application

As you saw in previous chapters, creating subscriptions with the SQL-NS Subscription Management API requires connections to the instance and application. These connections are represented by objects of the NSInstance and NSApplication classes. Listing 18.5 shows the code in the AddSubscriptions program that establishes an NSInstance object for the MusicStore instance and an NSApplication object for its SongAlerts application.

Listing 18.5. Code That Creates the NSInstance and NSApplication Objects in the AddSubscriptions Program

class Program
{
    ...
    private const string InstanceName = "MusicStore";
    private const string ApplicationName = "SongAlerts";
    ...
    private static AuthenticationMode authenticationMode;
    private static string sqlUserName;
    private static string sqlPassword;

    static int Main(string[] args)
    {

        ...
        try
        {
            // Create NSInstance and NSApplication objects.
            NSInstance nsInstance;
            NSApplication nsApplication;

            if (AuthenticationMode.WindowsAuthentication ==
                authenticationMode)
            {
                nsInstance = new NSInstance(InstanceName);
            }
            else
            {
                nsInstance = new NSInstance(
                    InstanceName,
                    sqlUserName,
                    sqlPassword);
            }

            nsApplication = new NSApplication(
                nsInstance,
                ApplicationName);

            ...
        }
       ...
    }
    ...
}

The code declares NSInstance and NSApplication objects as local variables and then proceeds to instantiate them. If Windows Authentication is used (as indicated by the value of the authenticationMode member variable, set based on the program's command-line arguments) the NSInstance object is instantiated with the constructor that takes only the instance name. If SQL Server Authentication is used, the NSInstance object is created with the constructor that takes the instance name as well as a SQL username and password. In both cases, the instance name value is obtained from a string constant.

After creating the NSInstance object, the code creates the NSApplication object. The NSApplication constructor takes a reference to the NSInstance object and the application name. Like the instance name, the application name is also obtained from a string constant.

Obtaining Needed SMO Objects

As described in Chapter 16, SMO is an API that provides programmatic access to database management capabilities. Classes in the SMO API can be used to manage SQL Server entities, such as databases, tables, and security roles. In addition to performing management operations, SMO classes can also be used to reflect over existing SQL Server entities. For example, SMO can be used to enumerate and discover the properties of the databases on a particular SQL Server, as well as the objects (such as tables, views, functions, and stored procedures) within each database.

In the SQL-NS Condition API, references to SQL Server entities are generally represented by instances of their corresponding SMO classes. For example, the FunctionValue class, which represents condition terms that use SQL functions, has a property that identifies the SQL function to call. This property is defined as an object of the SMO class UserDefinedFunction. Using the SMO API, we can obtain a UserDefinedFunction object for any SQL function in the database and use this to initialize a FunctionValue object.

Similarly, the PropertyRule and FilterRule classes have properties that specify the table or view to use as the input type. These are initialized using SMO Table or View objects. These Table and View objects can be obtained by enumerating the contents of a database using other SMO classes.

The code in Listing 18.6 shows creation of the SMO objects needed to construct conditions later in the AddSubscriptions program. For the sake of simplicity, these objects are all instantiated in one place, rather than spread throughout the code that builds the conditions.

Listing 18.6. Code That Creates SMO Objects in the AddSubscriptions Program

class Program
{

    ...
    private const string DatabaseName = "MusicStore";
    private const string SchemaName = "SongAlerts";
    ...
    private const string EventViewName = "SongAddedDetails";

    private const string CatalogSchemaName = "Catalog";
    private const string ArtistsTableName = "Artists";
    private const string SongsTableName = "Songs";
    private const string AlbumsTableName = "Albums";
    ...
    private static string sqlServer;
    private static AuthenticationMode authenticationMode;
    private static string sqlUserName;
    private static string sqlPassword;

    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            // Create SMO Server and Database objects.
            ServerConnection serverConnection = new ServerConnection();
            serverConnection.ServerInstance = sqlServer;

            if (AuthenticationMode.SqlServerAuthentication ==
               authenticationMode)
            {
                serverConnection.LoginSecure = false;
                serverConnection.Login = sqlUserName;
                serverConnection.Password = sqlPassword;
            }

            Server server = new Server(serverConnection);
            Database musicStoreDB = server.Databases[DatabaseName];

            // Obtain SMO objects for the database objects used
            // later when defining rules.
            View songAddedDetailsView =
                musicStoreDB.Views[EventViewName, SchemaName];

            Table songsTable =
                musicStoreDB.Tables[
                    SongsTableName,
                    CatalogSchemaName];

            Table artistsTable =
                musicStoreDB.Tables[
                    ArtistsTableName,
                    CatalogSchemaName];

            Table albumsTable =
                musicStoreDB.Tables[
                    AlbumsTableName,
                    CatalogSchemaName];
            ...
        }
        ...
    }
    ...
}

The code begins by creating an SMO ServerConnection object to represent a connection to the SQL Server. If SQL Server Authentication is used, the ServerConnection object is initialized with the required connection information. Based on the ServerConnection object, the code then creates a Server object to represent the server itself. This pattern is common to most SMO and NMO programs.

With the Server object established, the code obtains an SMO Database object representing the music store database. The object is obtained by accessing the Server object's Databases collection, using the database name as a key. Given this Database object, the code then proceeds to obtain four more SMO objects that represent tables and views within the music store database. Following a similar pattern, these objects are obtained from collections on the Database object, using the appropriate names as keys.

The four objects obtained represent the SongAddedDetails view (described in the section, "Creating the Input: The SongAddedDetails View," p. 612) and the songs, artists, and albums tables in the music store catalog. The View object is obtained from the Database object's Views collection and the Table objects are obtained from its Tables collection. When accessing the Tables and Views collections, both the object name and the name of its containing schema are given as keys.

As you'll see in subsequent sections, the SMO objects obtained here are used in the condition definitions that follow. Specifically, the View object representing SongAddedDetails view is used to define the input type for a property rule and the Table objects are used to define the input type for a filter rule and the join clauses in its associated link leaf condition.

Creating Subscription Objects

Each subscription that the AddSubscriptions program creates is represented by a Subscription object. For each subscription, the program creates the Subscription object, sets its basic properties, and then sets its condition. It then calls the Subscription object's Add() method to record the subscription in the database.

The creation of the subscription object and the setting of its basic properties (like the subscriber ID and the subscription fields) is the same for each subscription the program creates. For this reason, the creation and basic initialization of the subscription objects is broken out into a helper method. The main AddSubscriptions code calls this method to obtain a new Subscription object for each new subscription. Because the Subscription object obtained from the helper method is already initialized with the basic properties, the main AddSubscriptions code can simply focus on the setting of the subscription condition.

Listing 18.7 shows the implementation of the CreateSubscriptionObject() helper method. The code in this method is similar to the code for creating subscriptions that you saw in Chapter 7.

Listing 18.7. The CreateSubscriptionObject() Helper Method

class Program
{
    ...
    private const string SubscriptionClassName = "NewSongsOfInterest";
    private const string RuleName = "MatchNewSongsOfInterest";
    ...
    private static Subscription CreateSubscriptionObject(
        NSApplication nsApplication,
        string subscriberId,
        string deviceName,
        string locale)
    {
        Subscription subscription;

        subscription = new Subscription(
            nsApplication,
            SubscriptionClassName);
        subscription.SubscriberId = subscriberId;
        subscription.Enabled = true;
        subscription["DeviceName"] = deviceName;
        subscription["Locale"] = locale;

        subscription.RuleName = RuleName;
        return subscription;
    }
    ...
}

CreateSubscriptionObject() takes four arguments: an NSApplication object representing the application, the subscriber ID value to set in the Subscription object, and the values to use for the subscription's DeviceName and Locale fields. It creates a new Subscription object, passing the NSApplication object and the subscription class name (obtained from a string constant) to the Subscription constructor. It then sets the object's SubscriberId property to the value passed in, sets the Enabled property to true, and then sets the subscription fields to the given values.

Finally, the code sets the Subscription object's RuleName property. Setting the RuleName property associates the subscription with a particular rule in the subscription class. The specified rule's <ConditionAction> element defines the input type for the conditions that can be associated with the Subscription object. In this case, the RuleName property is set to the name of the event rule in our subscription class.

Note

Setting the RuleName property is necessary only if the Subscription object's Condition property will also be set. In Subscription objects that don't have user-defined condition logic (those that represent subscriptions based on developer-defined logic), neither RuleName nor Condition need to be set.

After its basic properties have been set, the Subscription object is returned. Notice that the helper method does not set the object's Condition property or call its Add() method. Both of those things are done outside the helper method, by the main AddSubscriptions code. The sections that follow illustrate how the Condition properties on Subscription objects created by this helper method can be set to a variety of condition objects.

Creating Subscriptions with Simple Conditions

To begin the series of condition examples, let's look at a subscription with a simple leaf condition. Listing 18.8 shows the code that creates such a subscription.

Listing 18.8. Creating a Subscription with a Simple Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
           ...
           Subscription subscription;

           subscription = CreateSubscriptionObject(
               nsApplication,
               "Bob",
               "DefaultDevice",
               "en-US");

           // Define condition:
           // ArtistName = "Miles Davis"
           subscription.Condition = new SimpleLeafCondition(
               new FieldValue("ArtistName"),
               SimpleOperator.Equals,
               new ConstantValue("Miles Davis"));

           // Add the subscription to the database.
           subscription.Add();
           ...
        }
        ...
    }
    ...
}

The code calls the helper method CreateSubscriptionObject() to create a new subscription object. A reference to this object is stored in the local variable subscription. As you saw in Listing 18.7, objects returned from CreateSubscriptionObject() have had their basic properties set. All that remains is to set the Condition property appropriately and then add the subscription to the database by calling its Add() method.

Note

The AddSubscriptions program does not create subscriber and subscriber device records. In our sample application, subscriber and subscriber device records are created via a T-SQL script. (Instructions for running this script are provided later in this chapter, in the section, "Testing the Music Store Application with User-Defined Logic," p. 643.) The script creates eight subscriber records, with the subscriber IDs "Bob", "Alex", "Mary", "Emily", "Joe", "Jane", "David", and "Lisa". It also creates an associated subscriber device for each subscriber named, "DefaultDevice". The AddSubscriptions program assumes the existence of these subscriber and subscriber device records.

In Listing 18.8, the Subscription object's Condition property is set to a new SimpleLeafCondition object. The SimpleLeafCondition constructor takes three arguments: left and right operands and a relational operator. In this example, the left operand is a new FieldValue object, referencing the ArtistName field in the input data. The operator is the relational operator, equals. This is indicated by means of the Equals member of the SimpleOperator enumeration. The condition's right operand is a ConstantValue object representing the literal string, "Miles Davis". Thus, the SimpleLeafCondition implements the condition expression ArtistName = "Miles Davis".

After the condition is defined, the Subscription object's Add() method is called. Internally, the Add() method validates the condition and records all the subscription information (including the condition definition) in the database.

Notice how the FieldValue and ConstantValue objects in Listing 18.8 are created inline with the call to the SimpleLeafCondition constructor. This is a common pattern in code that constructs conditions: subobjects that make up a condition expression are often created directly within the statement that creates the main condition object.

All condition classes also support an alternative initialization pattern: they can be instantiated with parameterless constructors and then initialized via their various public properties. For example, the condition in Listing 18.8 could have been created by instantiating a SimpleLeafCondition object with its parameterless constructor and then setting the object's Left, Operator, and Right properties. However, this technique requires you to remember which properties need to be set to fully initialize the object. It's easy to create an object and then forget to initialize one of its properties, leading to a failure at the time the subscription is added to the database. I personally believe the method shown in Listing 18.8 (creating and initializing the condition object with a constructor that takes parameters) is less error prone. The signature of the constructor defines the required initialization parameters; failure to provide any of these parameters will result in an error at compile time.

Creating Subscriptions with Compound Conditions

In this section, we look at some examples of compound conditions. Compound conditions combine simpler conditions with the logical operators AND, OR, and NOT. Listing 18.9 shows the code that creates a compound condition that combines two simple leaf conditions with the OR operator.

Listing 18.9. Creating a Subscription with a Compound OR Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
           ...
           subscription = CreateSubscriptionObject(
               nsApplication,
               "Alex",
               "DefaultDevice",
               "en-US");

           // Define condition:
           // Genre = "Jazz" OR Genre = "Hip Hop"
           subscription.Condition = new OrCondition(
               new SimpleLeafCondition(
                   new FieldValue("Genre"),
                   SimpleOperator.Equals,
                   new ConstantValue("Jazz")),
               new SimpleLeafCondition(
                   new FieldValue("Genre"),
                   SimpleOperator.Equals,
                   new ConstantValue("Hip Hop")));

           // Add the subscription to the database.
           subscription.Add();
           ...
        }
        ...
    }
    ...
}

Like before, a new subscription object is obtained by calling the CreateSubscriptionObject() helper method. The new Subscription object's Condition property is then set to a new OrCondition object. The OrCondition constructor takes a variable length list of parameters, each of which is a condition object to include in the OR expression. In this example, two SimpleLeafConditions are passed to the OrCondition constructor. These SimpleLeafCondition objects are created inline with the call to the constructor. Like the example in the previous section (shown in Listing 18.8), each simple leaf condition in this example compares a field of the input to a constant value, with the equality operator.

The condition object created in Listing 18.9 expresses the condition (Genre = "Jazz") OR (Genre = "Hip Hop"). This is the program's first example of a subscription that could not have been expressed in the previous version of the music store application.

The code in Listing 18.10 creates another compound condition. This time, two simple leaf conditions are combined with the AND operator.

Listing 18.10. Creating a Subscription with a Compound AND Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            subscription = CreateSubscriptionObject(
                nsApplication,
                "Emily",
                "DefaultDevice",
                "en-US");

            // Define condition:
            // ArtistName = "Dido" AND AlbumTitle = "Life for Rent"
            subscription.Condition = new AndCondition(
                new SimpleLeafCondition(
                    new FieldValue("ArtistName"),
                    SimpleOperator.Equals,
                    new ConstantValue("Dido")),
                new SimpleLeafCondition(
                    new FieldValue("AlbumTitle"),
                    SimpleOperator.Equals,
                    new ConstantValue("Life for Rent")));

            // Add the subscription to the database.
            subscription.Add();
            ...
        }
       ...
    }
    ...
}

The condition object assigned to the Subscription object's Condition property is an instance of the AndCondition class. Like the OrCondition we looked at in the previous example, the AndCondition constructor takes a variable number of condition objects. The AndCondition object represents the compound condition formed by combining the conditions expressed by these child condition objects with the logical AND operator. The code in Listing 18.10 creates an AndCondition condition object that expresses the condition (ArtistName = "Dido") AND (AlbumTitle = "Life For Rent").

Listing 18.11 shows an example of a compound condition based on the NOT operator. It also illustrates how compound conditions can be composed.

Listing 18.11. Creating a Subscription with a NOT Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            subscription = CreateSubscriptionObject(
                nsApplication,
                "Joe",
                "DefaultDevice",
                "en-US");

            // Define condition:
            // NOT(ArtistName = "Dido"
            //     AND AlbumTitle = "Life for Rent"))
            subscription.Condition = new NotCondition(
                new AndCondition(
                    new SimpleLeafCondition(
                        new FieldValue("ArtistName"),
                        SimpleOperator.Equals,
                        new ConstantValue("Dido")),
                    new SimpleLeafCondition(
                        new FieldValue("AlbumTitle"),
                        SimpleOperator.Equals,
                        new ConstantValue("Life for Rent"))));

            // Add the subscription to the database.
            subscription.Add();
            ...
        }
        ...
    }
    ...
}

In Listing 18.11, the Subscription object's Condition property is set to a new NotCondition object. The NotCondition constructor takes a single child condition object representing the condition expression to be negated. Because the type of the NotCondition constructor's parameter is the base class Condition (from which all condition classes are derived) it can take a reference to any condition objecteven another compound condition. Here, the child condition is an AndCondition object. In fact, the definition of this AndCondition object is the same as the one in the previous example. Therefore, the condition created in Listing 18.11 is the negation of the condition created in Listing 18.10. The subscription created in Listing 18.11 will be matched against precisely those songs that do not match the subscription in Listing 18.10.

Note

Listing 18.11 actually creates four condition objects in one statement. It's important to understand the structure of this statement to see how these condition objects form the final condition expression.

At the top level is the NotCondition object. Its constructor takes a single child condition object. In this example, because the child condition is an AndCondition, created inline within the call to the NotCondition constructor, it may appear as though more than one condition is being passed to the NotCondition constructor. This is not the case; the NotCondition constructor takes just the one AndCondition object. The AndCondition constructor is passed two child conditions (both of them SimpleLeafCondition objects). These are also created inline. The two SimpleLeafCondition objects are children of the single AndCondition object, which is the only child of the NotCondition object.

Using Function and Expression Values

In all the examples so far, only constant values and field values have been used. In this section, we look at an example of a condition defined with function values and expression values.

The code in Listing 18.12 creates a condition expression that compares the result of a function call with the result of an arithmetic expression. This is a somewhat contrived example, but it illustrates how these kinds of values can be used.

Listing 18.12. Using Function and Expression Values to Define a Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            subscription = CreateSubscriptionObject(
                nsApplication,
                "Jane",
                "DefaultDevice",
                "en-US");

            // Define condition:
            // NumberOfWords(ArtistName) >= 1+2
            subscription.Condition = new SimpleLeafCondition(
                new FunctionValue(
                    musicStoreDB.UserDefinedFunctions[
                        "NumberOfWords",
                        CatalogSchemaName],
                    new FieldValue("ArtistName")),
                SimpleOperator.GreaterThanOrEqualTo,
                new ExpressionValue(
                    new ConstantValue(1),
                    ExpressionOperator.Addition,
                    new ConstantValue(2)));

            // Add the subscription to the database.
            subscription.Add();
            ...
        }
        ...
    }
    ...
}

This code sets the Subscription object's Condition property to a SimpleLeafCondition object. Structurally, this simple leaf condition is like the ones you've seen in previous examples: it compares one value to another with a relational operator. The difference is that in this example, the values are not just constants or field values.

Like before, the two values are created inline with the call to the SimpleLeafCondition constructor. The first value is an object of the FunctionValue class. The FunctionValue constructor takes the SMO UserDefinedFunction object representing the function to call. This is obtained from the UserDefinedFunctions collection on the Database object created earlier (see the section "Obtaining Needed SMO Objects," p. 622). In this example, the function name is NumberOfWords and it's defined in the music store catalog schema. Both the function name and schema name are used as keys into the Database object's UserDefinedFunctions collection.

The function, NumberOfWords, is defined in the script that creates the music store database. It takes a string argument and returns a count of the number of words in it. For example, given the string "Life for Rent", the function would return the number 3. It's not important to go into the implementation of this function, but if you're curious, you can look for it in the script C:\SQL-NS\Samples\UserDefinedLogic\CreateMusicStore.sql.

In addition to the UserDefinedFunction object that identifies the SQL function to call, the FunctionValue constructor takes a variable length list of Argument objects that specify values for the function's arguments. Because all value classes derive from the base class Argument, any value object can be specified in this list. The NumberOfWords function expects a single argument, so the FunctionValue constructor is passed just one value object in this example. This is a FieldValue object referencing the ArtistName field. Thus, our FunctionValue object represents a call to the NumberOfWords function, passing the value of the ArtistName field in the input data as the function's argument. When evaluated, this returns the number of words in the artist name.

The operator specified as the second argument to the SimpleLeafCondition constructor in Listing 18.12 is the GreaterThanOrEqualTo member of the SimpleOperator enumeration. This corresponds to the relational operator greater than or equal to (>=).

The third argument passed to the SimpleLeafCondition constructor represents the right operand in the condition expression. In this example, an ExpressionValue object is passed. The ExpressionValue constructor takes two value objects to use as operands and an arithmetic operator. From these, it forms an arithmetic expression. The constructor's operand arguments are of type Argument, so any value object can be used. Here, two ConstantValue objects, representing the constants 1 and 2, respectively, are passed. The arithmetic operator is specified using the ExpressionOperator enumeration. In this example, the Addition member of the enumeration is used, which indicates that the two operands should be added together.

By assembling all the pieces described here, our subscription's condition object represents the expression NumberOfWords(ArtistName) >= (1 + 2). This will match all songs in which the artist name consists of three or more words. As mentioned earlier, this is a contrived example, but it does illustrate how function and expression values can be specifiedand how user-defined logic can provide users a way to create some really strange subscriptions!

Creating User-Defined Rules to Build Conditions

This section shows the use of user-defined rules to create conditions. Two examples are shown: one that uses a property rule and another that uses a filter rule.

Although these examples each create only one condition with each rule, remember that user-defined rules represent reusable logic. A single rule can be incorporated into several different conditions. This allows the logic defined by the rule to be reused without being duplicated in each condition definition.

Creating Property Rules

As described in the earlier section "Property Rules" (p. 599), a property rule computes a scalar-valued result from a row of input data. The results of property rules can be treated like computed properties of the input over which they're defined. Property rules are created independently and can be used as terms in one or more condition expressions.

Earlier, we looked at a conceptual example of a property rule that defines a CanDance property of songs. Listing 18.13 shows code that actually creates this property rule using classes in the Condition API and then uses it in a subscription condition.

Listing 18.13. Using a Property Rule in a Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            subscription = CreateSubscriptionObject(
                nsApplication,
                "David",
                "DefaultDevice",
                "en-US");

            // The condition for this subscription uses a property
            // rule, so we construct that rule here.

            // Create the rule that defines the  "canDance"
            // property.
            PropertyRule canDanceRule = new PropertyRule();
            canDanceRule.InputType =
                new ViewInputType(songAddedDetailsView);

            // Add statement that says the value is true, if the
            // Genre = "Hip Hop".
            canDanceRule.PropertyStatements.Add(
                new PropertyStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Genre"),
                        SimpleOperator.Equals,
                        new ConstantValue("Hip Hop")),
                    new ConstantValue(true)));

            // Add statement that says the value is false, if the
            // Genre = "Rock".
            canDanceRule.PropertyStatements.Add(
                new PropertyStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Genre"),
                        SimpleOperator.Equals,
                        new ConstantValue("Rock")),
                    new ConstantValue(false)));

            // Add statement that says the value is false, if the
            // Genre = "Jazz".
            canDanceRule.PropertyStatements.Add(
                new PropertyStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Genre"),
                        SimpleOperator.Equals,
                        new ConstantValue("Jazz")),
                    new ConstantValue(false)));

            // Define condition:
            // canDance property rule's value is true.
            subscription.Condition = new BooleanLeafCondition(
                new RuleValue(canDanceRule));

            // Add the subscription to the database.
            subscription.Add();

            ...
        }
        ...
    }
    ...
}

Property rules are represented by objects of the PropertyRule class. The code in Listing 18.13 creates a new PropertyRule object for our CanDance rule and specifies the rule's input type by setting the object's InputType property.

Property rules must have the same input type as the conditions in which they are used. Because our subscription conditions use the SongAddedDetails view as their input type, this view must also be the input type for our property rule. This is specified by setting the PropertyRule object's InputType property to a new ViewInputType object, initialized with the SMO View object for the SongAddedDetails view. This SMO View object was obtained earlier, in the code shown in Listing 18.6.

ViewInputType is one of two classes in the Condition API that can be used to specify input types for user-defined rules. As this example shows, its constructor takes an SMO View object representing the input view. The other input type class is TableInputType, which, as its name suggests, is used to represent inputs that are tables. The TableInputType class is used in the filter rule example in the next section.

After specifying the rule's input type, the code in Listing 18.13 defines the rule's three property statements. It creates a new PropertyStatement object for each statement and adds it to the PropertyStatements collection on the PropertyRule object. This associates the statements with the rule. The PropertyStatement objects are created inline with the call to the collection's Add() method.

Caution

Remember that statements in a property rule are ordered: if more than one statement's condition evaluates to true for a given input, the result value is taken from the earliest statement. Therefore, the order in which PropertyStatement objects are added to the PropertyStatements collection is significant.

Recall from the "Property Rules" section (p. 599) that property statements have the following form:

IF <Condition> THEN <Value>

In the API, this form is reflected in the PropertyStatement objects: each PropertyStatement object contains a condition object and a result value object. These can be the same kinds of condition and value objects used in subscription conditions. In Listing 18.13, each PropertyStatement object is created with a SimpleLeafCondition object (expressing a simple predicate over the Genre field in the input) and a ConstantValue object representing a Boolean constant result. These condition and value objects are created within the call to the PropertyStatement constructor.

In this example, the same kind of condition is used in each statement, but this is not a requirement. Different statements in a property rule can have different kinds of condition objects. The same is true of the statements' result value objects. For example, some statements may have ConstantValue result objects and others may have FieldValue or ExpressionValue result objects.

Property rules can be referenced in condition expressions by means of RuleValue objects. RuleValue is a value class, derived from the Argument base class, like all other value classes. A RuleValue object is initialized with a PropertyRule object and then represents the result of that property rule.

Because RuleValue is just another value class, RuleValue objects can be used in any place a value object is needed. Depending on the type of value returned by the particular property rule, some contexts make more sense than others. In this example, the property rule returns Boolean values, so it makes sense to use the RuleValue object where Booleans are expected. Had we defined a property rule that returns integers, it could have been referenced in RuleValue objects within arithmetic expressions.

The code in Listing 18.13 uses our property rule to create a Boolean leaf condition for the subscription. The condition object is an instance of the BooleanLeafCondition class, which represents condition expressions that consist of a single Boolean value. BooleanLeafCondition objects can be initialized with any value object that represents a Boolean value. Here the value object is a RuleValue object initialized with our CanDance property rule. Because the property rule returns Booleans, it can be used within a BooleanLeafCondition.

Note

The equivalent condition logic could have been achieved by creating a SimpleLeafCondition that compares our property rule's value to the constant value, true. The BooleanLeafCondition is a slightly more compact way of representing the same logic.

As this example shows, after the property rule is defined, its value can be used like a field of the input. And because property rules are reusable, we can use the values they compute in several subscriptions or in the definition of other rules. To do this in code, we could create additional RuleValue objects that all refer to the same PropertyRule object.

Creating Filter Rules and Link Leaf Conditions

The earlier section "Filter Rules and Link Leaf Conditions" (p. 601) describes how filter rules can be used with link leaf conditions to express logic over data in tables or views other than a condition's input. The example described in that section was based on a stock notification application.

In this section, we look at an example of a filter rule and link leaf condition within the context of the music store application. In this example, we create a subscription that matches all songs by a subscriber's favorite artists. Instead of hard-coding the logic that determines which artists are favorites directly into the subscription condition, we can break it out into a filter rule. This filter rule is defined over the Artists table in the music store catalog schema. The filter rule's statements define which rows in the Artists table should be included in the set of favorites, based on a number of conditions.

The filter rule is then used in a link leaf condition in the subscription. The link leaf condition specifies a set of join clauses that link the condition's input, the SongAddedDetails view, with the input to the filter rule, the Artists table. Given a particular input row, if any of the related rows in the Artists table are in the set of favorites defined by the filter rule, the condition is satisfied.

Admittedly, this isn't the most compelling example because the only information in the Artists table is the artist name and this is already available for each song in the SongAddedDetails view. The example would be more compelling if the Artists table contained other properties of artists that could be used in the filter rule's conditions to define what it means for an artist to be a favorite. Regardless, this example still serves to illustrate the use of the Condition API to create filter rules and link leaf conditions. Listing 18.14 shows the corresponding code.

Listing 18.14. Using a Filter Rule with a Link Leaf Condition

class Program
{
    ...
    static int Main(string[] args)
    {
        ...
        try
        {
            ...
            subscription = CreateSubscriptionObject(
                nsApplication,
                "Lisa",
                "DefaultDevice",
                "en-US");

            // Create a rule that defines my favorite artists.
            FilterRule favoriteArtists = new FilterRule();
            favoriteArtists.InputType =
                new TableInputType(artistsTable);

            // Add a statement that includes Miles Davis.
            favoriteArtists.FilterStatements.Add(
                new FilterStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Name"),
                        SimpleOperator.Equals,
                        new ConstantValue("Miles Davis")),
                    FilterAction.Include));

            // Add a statement that includes Black Eyed Peas.
            favoriteArtists.FilterStatements.Add(
                new FilterStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Name"),
                        SimpleOperator.Equals,
                        new ConstantValue("Black Eyed Peas")),
                    FilterAction.Include));

            // Add a statement that excludes Dido.
            favoriteArtists.FilterStatements.Add(
                new FilterStatement(
                    new SimpleLeafCondition(
                        new FieldValue("Name"),
                        SimpleOperator.Equals,
                        new ConstantValue("Dido")),
                    FilterAction.Exclude));

            subscription.Condition = new LinkLeafCondition(
                LinkLeafOperator.Any,
                favoriteArtists,
                new JoinClause(
                    songAddedDetailsView.Columns["SongId"],
                    songsTable.Columns["Id"]),
                new JoinClause(
                    songsTable.Columns["AlbumId"],
                    albumsTable.Columns["Id"]),
                new JoinClause(
                    albumsTable.Columns["ArtistId"],
                    artistsTable.Columns["Id"]));

            // Add the subscription to the database.
            subscription.Add();

            ...
        }
        ...
    }
    ...
}

Filter rules are represented by instances of the FilterRule class. This code creates a new FilterRule object for our filter rule and sets its InputType property. The input type for the rule is the Artists table, so the InputType property is set to a TableInputType object, initialized with the SMO Table object for the Artists table. This SMO Table object was obtained earlier, in the code shown in Listing 18.6.

The code then creates FilterStatement objects to represent the statements in the filter rule definition. Each FilterStatement object is added to the FilterStatements collection on the FilterRule object.

Filter rule statements have the following form:

IF <Condition> THEN <Action>

In the API, this form is reflected in the FilterStatement class, which defines Condition and Action properties. A FilterStatement object's Condition property can be set to any of the supported condition objects in the Condition API. Its Action property can be set to the values defined in the FilterAction enumeration: Include and Exclude.

The filter rule in this example consists of the following three statements:

IF Name = "Miles Davis" THEN Include
IF Name = "Black Eyed Peas" THEN Include
IF Name = "Dido" THEN Exclude

In the Artists table, the artist names are stored in the Name column. The conditions in these statements are therefore implemented by comparing the Name field in each input row with a particular artist name. The actions specify whether to include or exclude rows with those particular artist names. The first two statements specify that the artists Miles Davis and Black Eyed Peas should be included. The last statement says that the artist Dido should be excluded.

In Listing 18.14, the condition object and action value for each statement are specified as arguments to the FilterStatement constructor. The statements' conditions are all SimpleLeafCondition objects. Their actions are specified using the members of the FilterAction enumeration.

After creating all the filter rule's statements, the code creates the LinkLeafCondition object in which the filter rule is used. The LinkLeafCondition constructor takes the link operator (specified as a member of the LinkLeafOperator enumeration), the FilterRule object, and a series of JoinClause objects representing the join clauses that link the condition's input to the filter rule's input. The LinkLeafCondition constructor accepts a variable number of JoinClause objectsyou can pass as many as are needed to establish the link between the two inputs.

In this example, the link operator ANY is specified using the LinkLeafOperator.Any value. Because the relationship between the SongAddedDetails view and the Artists table is one-to-one (every song has only one artist in our simplified music store schema), we could also have used the LinkLeafOperator.Every operator without changing the logic of the condition.

The FilterRule object passed to the LinkLeafCondition constructor is the one we just created. A reference to the FilterRule was kept in the local variable favoriteArtists. This local variable is now passed as the second argument to the LinkLeafCondition constructor. If we needed to create subsequent subscription conditions that refer to the same filter rule, we could reuse the same FilterRule object.

The join clauses passed to the LinkLeafCondition constructor link the SongAddedDetails view to the Artists table via the Songs and Albums tables. The link chain proceeds as follows:

  1. The SongAddedDetails view is joined with the Songs table based on song ID. In the SongAddedDetails view, the song ID column is called SongId; in the Songs table, it is called Id.

  2. The Songs table is joined with the Albums table on album ID. In the Songs table, the join column is AlbumId; in the Albums table, it is Id.

  3. The Albums table is joined with the Artists table on artist ID. The join column in the Albums table is ArtistId; in the Artists table, it is Id.

The JoinClause objects, created inline during the call to the LinkLeafCondition constructor, are each initialized with a pair of columns. In each pair, the first column is in the current table in the link chain and the second column is in the next table in the chain. Because the chain starts with the condition's input, the first column is from the SongAddedDetails view. The columns are specified as SMO Column objects, obtained from the Columns collections on the various SMO View and Table objects we created earlier.

The LinkLeafCondition object is assigned to the Subscription object's Condition property and then the Subscription object's Add() method is called. In the Add() method, the Subscription object validates all the elements of the link leaf condition, including the filter rule and the join clauses, before recording them in the application database.

Security Configuration for User-Defined Logic

In this section, we look at the configuration steps required to implement a low-privilege sandbox for user-defined logic in our sample application. As you saw in Listing 18.3, the <SqlLogin> element in the event rule's <ConditionAction> declaration specifies the value "LowPrivEvaluator". When the SQL-NS compiler creates the user account it will use for impersonation during condition evaluation, it will associate the account with a server login of this name. Therefore, we need to create this login before compiling the application.

In our sample application, the server login is created by the script C:\SQL-NS\Samples\UserDefinedLogic\CreateLowPrivEvaluatorLogin.sql. One of the setup steps in the instructions for testing the sample invokes a batch file that executes this script. Listing 18.15 shows the code in this script.

Listing 18.15. Creating a Low-Privileged Login

USE [Master]
GO
IF NOT EXISTS (
    SELECT name
    FROM sys.syslogins
    WHERE name = 'LowPrivEvaluator'
)
    CREATE LOGIN [LowPrivEvaluator] WITH PASSWORD='lowPrivilege1234!'
GO

This code first checks whether a login with the name "LowPrivEvaluator" already exists. It creates the login, using a CREATE LOGIN statement, only if the login doesn't already exist. The login is created with a password, meaning that it authenticates via SQL Server Authentication. Even if your system generally uses Windows Authentication, configuring this low privileged login to use SQL Server Authentication keeps things simple. To use Windows Authentication, you would need a Windows account to associate with the login.

Caution

The password for the LowPrivEvaluator login is hard-coded in the CreateLowPrivEvaluatorLogin.sql script. Although this account is intended to have very low privileges, you may still want to remove the password from the script (after executing it, as you'll be instructed to do in the following section) or change it to a different value. Note that the login created by this script is dropped by the sample's cleanup script.

At compile time, the SQL-NS compiler checks for the existence of the login specified in the <SqlLogin> element. Creating the server login ensures that the application will compile successfully. During the compilation, the compiler will create a user account in the application's database with the username specified in the <SqlUser> element of the <ConditionAction> declaration. In our sample application, the <SqlUser> element specifies the username LowPrivEvaluator. Initially, this account will have no permissions (the compiler does not grant it any). After the application is compiled, we must explicitly grant this account the permissions it needs. This must be done before the application is started, otherwise condition evaluation will fail.

Note

In this example, the user account and the server login with which it is associated have the same name, LowPrivEvaluator. This is not a requirementyou may specify different values for the <SqlLogin> and <SqlUser> elements.

In general, the user account specified in a <ConditionAction> element's <SqlUser> subelement needs only SELECT permissions on the tables and views referenced in subscription conditions. These tables and views include the condition input (as specified in the <InputName> element within the <ConditionAction>), as well as the inputs of any property rules and filter rules referenced in the subscription conditions. In addition, SELECT permissions are needed on all the tables and views referenced in join clauses in any link leaf conditions.

In the case of our application, the SongAddedDetails view is the input type for the subscription conditions and the one property rule. The AddSubscriptions program defines a filter rule over the music store catalog's Artists table, and the join clauses in the link leaf condition that consumes this filter rule reference the Songs and Artists tables. Therefore, we need to grant the condition evaluation user account SELECT permissions on the SongAddedDetails view, as well as the Songs, Albums, and Artists tables.

Permissions are granted in the sample's SQLPermissions.sql script. This script includes the statements that grant the usual permissions to the SQL-NS engine account, as well as those that grant permissions to the LowPrivEvaluator account. The latter statements are shown in Listing 18.16.

Listing 18.16. Granting Permissions to the Low Privilege User

GRANT SELECT ON [SongAlerts].[SongAddedDetails] TO [LowPrivEvaluator]
GRANT SELECT ON [Catalog].[Songs] TO [LowPrivEvaluator]
GRANT SELECT ON [Catalog].[Albums] TO [LowPrivEvaluator]
GRANT SELECT ON [Catalog].[Artists] TO [LowPrivEvaluator]

When we test the application in the following section, one of the steps you'll run invokes the SQLPermissions.sql script to grant all these permissions.

Testing the Music Store Application with User-Defined Logic

Now that you've seen all the important code in the application and the associated configuration scripts, it's time to compile the application and test it with some sample data. Use the following instructions to do this:

1.
Open a Notification Services Command Prompt and navigate to the scripts directory for the old version of the music store application, C:\SQL-NS\Samples\MusicStore\Scripts.

cd /d C:\SQL-NS\Samples\MusicStore\Scripts

2.
Run cleanup.cmd to remove any previous version of the music store application that you may have created on your system in previous chapters. This is necessary because the new version of the sample in this chapter uses the same instance and application name as the old version.

3.
Navigate to the scripts directory for the new version of the sample, C:\SQL-NS\Samples\UserDefinedLogic\Scripts.

cd /d C:\SQL-NS\Samples\UserDefinedLogic\Scripts

4.
Run create_low_priv_evaluator_login.cmd. This executes the statements shown in Listing 18.15 that create the low-privilege login if it doesn't already exist.

5.
Run create_music_store_database.cmd to create the music store database and populate it with data.

6.
Run create.cmd to create the SQL-NS instance.

7.
Run register.cmd to register the instance.

8.
Run enable.cmd to enable the instance.

9.
Run grant_permissions.cmd to grant the required database permissions, including those needed by the low-privilege user account. This executes the statements shown in Listing 18.16.

10.
Start the SQL-NS Windows service for the instance by issuing the following command:

net start NS$MusicStore

11.
Execute the SQL script C:\SQL-NS\Samples\UserDefinedLogic\AddSubscribers.sql in Management Studio. This creates the subscriber and subscriber device records.

12.
If you have not already opened the AddSubscriptions Visual Studio solution, open it now. The solution file is C:\SQL-NS\Samples\UserDefinedLogic\AddSubscriptions\AddSubscriptions.sln.

13.
Build the AddSubscriptions solution in Visual Studio.

14.
Return to the command prompt and invoke addsubscriptions.cmd from the sample's scripts directory (C:\SQL-NS\Samples\UserDefinedLogic\Scripts). This batch file invokes the AddSubcriptions program with the appropriate command-line arguments for your environment.

15.
Execute the SQL script C:\SQL-NS\Samples\UserDefinedLogic\SubmitEvents.sql in Management Studio. This submits a set of test events.

16.
Watch the notifications output file C:\SQL-NS\Samples\UserDefinedlogic\FileNotifications.txt. Within 30 seconds, a set of notifications will be written to this file.

The notifications written to the file are generated based on the user-defined conditions created by the AddSubscriptions program. Notice that the notifications look like the ones generated by the previous version of the music store application. This shows that the modifications introduced in this chapter have only changed the mechanism used to determine whether an event matches a subscription. The rest of the application, including the processing of generated notifications, works as it did before.

Note

You can verify that the correct notifications were generated by checking the event data submitted in step 15 with the conditions defined in Listings 18.8 through 18.14. If you look at the SubmitEvents.sql script, you'll see that it submits events with song IDs 1, 6, 12, and 14. You can query the SongDetails view in the music store catalog to see the data associated with these song IDs.

The AddSubscriptions program creates only one subscription for each subscriber. You can correlate the subscription conditions defined in Listings 18.8 through 18.14 with the generated notifications based on subscriber ID. Compare the subscriber ID value passed to the CreateSubscriptionObject() helper method in each listing with the subscriber IDs written to the output file with each notification.

Note

When you're done working with the sample instance, you can remove it from your system by running the cleanup.cmd script from the scripts directory C:\SQL-NS\Samples\UserDefinedLogic\Scripts.


Summary

SQL-NS applications generate notifications by matching subscriptions against events or chronicled data. The logic that determines when a match occurs is usually specified by developers when they create the applications. SQL-NS also offers an alternative model in which the matching logic can be specified by end users at the time they create their subscriptions. This model of user-defined logic is covered in this chapter.

User-defined logic is specified in terms of conditions on a subscription's input. The input is usually either a single event or a single row in a chronicle table. The SQL-NS condition model defines a few simple condition constructs that can be composed in various ways to create complex conditions.

Building a SQL-NS application that supports user-defined logic involves defining special subscription classes in the ADF. The rules in these subscription classes specify the kinds of conditions that can be used to express matching logic. They also define how notification rows are generated based on matches determined by user-defined logic.

The SMIs for applications that support user-defined logic provide users with a way to specify conditions when they create subscriptions. The user interface controls for creating conditions vary from application to application. However, all SMIs use the SQL-NS Condition API, an extension to the standard Subscription Management API, to represent conditions programmatically and store them in the application's database.

This chapter provides a detailed description of the techniques used to build SQL-NS applications that support user-defined logic. It explains the ADF elements that configure the use of user-defined logic, the details of the condition model itself, as well as the Condition API. All the concepts are first presented in general terms and then shown in practical use in a functional sample application.


Part V: Appendix

IN THIS PART


Appendix A. Creating a Development Environment for Your Own SQL-NS Applications

In Chapter 2, "Getting Set Up," you set up the development environment used to build the sample applications in this book. This development environment consists of a directory structure and a set of batch files that invoke SQL-NS commands, grant database and filesystem permissions, and set up auxiliary databases. Using the batch files means that you don't have to remember the exact syntax of every SQL-NS command or the precise sequence of steps required to create and manage the sample applications.

When you build SQL-NS applications of your own, it's a good idea to use a similar development environment. Doing so not only makes the development process easier, but also less error prone. Instead of designing a development environment from scratch, you can extend the one used for the sample applications in this book. This appendix explains how to do this.


The Structure of the Development Environment

Before learning how to extend the development environment, it's important that you understand how it works. This section explains the directory layout and some of the files involved. The environment you create for your own applications will have the same structure.

The Development Environment Directory Layout

We refer to the directory into which you copied the source files from this book's CD-ROM as the source code base directory. Throughout this book, the name of this directory is assumed to be C:\SQL-NS, although you can give it any name you choose.

The following directories exist under the source code base directory:

  • The Common directory contains files shared by the code for all the sample SQL-NS instances.

  • The Chapters directory contains supplementary files and scripts referred to in this book's various chapters.

  • The Samples directory contains the source code and build scripts for all the sample SQL-NS instances described in this book.

  • The Tools directory contains the executable files and source code for the custom tools used in the setup of the development environment and in some of the instructions in various chapters.

The Samples directory has a subdirectory called Minimal, which contains the source files for a minimal SQL-NS instance with one application. When you create a new SQL-NS instance, you begin by copying source code for this minimal instance. This gives you a working skeleton of a SQL-NS instance and application to which you can add your own code. The section "Extending the Development Environment" (p. 651) describes the exact steps in this process.

Each of the sample subdirectories within the Samples directory contains the code for a complete SQL-NS instance. Throughout the remainder of this appendix, I refer to these directories as instance directories because each one encapsulates the source code for an instance. The directories you create for new SQL-NS instances of your own will have a structure similar to these sample instance directories, so it's important to understand their subdirectory layout. The following subdirectories exist beneath all sample instance directories:

  • The Scripts subdirectory contains the batch files used to create, update, delete, register, unregister, enable, and disable the instance, as well as grant the required database and filesystem permissions.

  • A unique application subdirectory for each application in the instance contains the applications' source files. The name of each application subdirectory is the name of the application it contains. For example, in the MusicStore instance, the application name is SongAlerts, so there is a SongAlerts subdirectory.

The directories you create for your instances will also have a Scripts subdirectory containing the build scripts and an application directory for each application.

Important Files in the Development Environment

In the Common directory under the source code base directory, the setenv.cmd file sets up the environment variables used by the build scripts in all the instance directories. These environment variables provide the location of the source code base directory, the name of the SQL Server, the name of the service user, the SQL authentication mode, and other required information. When you completed the instructions given in the "Environment Variables" section (p. 36) in Chapter 2, you edited setenv.cmd to customize the environment variables for your system.

Each instance directory also contains a file called setenv.cmd in its Scripts subdirectory. All the build scripts (batch files that invoke nscontrol create, nscontrol update, and so on) in the Scripts subdirectory call this local setenv.cmd file before invoking any operations. The local setenv.cmd file in the Scripts subdirectory first calls the common setenv.cmd file (in the Common directory, one level up) and then sets up additional environment variables specific to the instance. This establishes the complete environment, including common and instance-specific parts, required by the commands in the build scripts.

Note

Most of the build scripts in the Scripts directory are explained in detail in Chapter 4, "Working with SQL-NS Instances."

As long as the environment variables in the local and common setenv.cmd files are set correctly, the build scripts will work for any instance. When you create a new instance, you usually don't have to change the common setenv.cmd file. However, you will have to create an instance-specific setenv.cmd file in the Scripts subdirectory of the instance directory to establish the instance-specific part of the environment for building the new instance. The "Creating the Directories and Files for a New Instance" section (p. 652) later in this appendix provides instructions for doing this.


Extending the Development Environment

This section describes how to extend the development environment used in this book to allow you to build new instances and applications of your own. The instructions in this section walk you through setting up the code and build scripts for a new SQL-NS instance with one application. You can follow this same procedure whenever you work on a new SQL-NS project. Extending this book's development environment, instead of designing your own environment from scratch, has the advantage that you can continue to use the same scripts and techniques you've become familiar with as you've worked through the examples in this book.

Caution

The instructions in this section assume you have set up the development environment for this book's samples according to the instructions given in Chapter 2. It's especially important that you successfully complete the instructions in the section "Customizing the Source Files for Your Environment," p. 39, in Chapter 2 before you proceed with the instructions given here.

Note that any new SQL-NS instances you create following the instructions in this section will have the same configuration as this book's samples, including the accounts used to run the SQL-NS engine and connect to the SQL Server.

Creating a Root Directory for New Instances

Assuming you do not want to create your new instances alongside this book's sample instances in the Samples directory, you should create a new top-level directory beneath C:\SQL-NS. Within this new directory, you can later create subdirectories for each instance you develop.

You can choose any name for the new directory, but for the purposes of the instructions given here, I will assume it's called NewInstances (if you choose a different name, replace NewInstances appropriately in all the following instructions). This directory must be created immediately beneath C:\SQL-NS, at the same level as the Samples directory.

You can create the directory for your new instances with the following command:

mkdir C:\SQL-NS\NewInstances

Alternatively, if you don't mind creating new instances directly in the Samples directory, you do not need to create a new directory. Simply replace C:\SQL-NS\NewInstances with C:\SQL-NS\Samples in all the subsequent instructions.

Creating the Directories and Files for a New Instance

Before creating the directories and files for a new instance, you must decide on the instance name and the name of the application in it. In the instructions given in this section, the instance name is assumed to be MyNewInstance, and the single application in the instance is assumed to be called MyNewApplication. When you invoke the commands described here to create a new instance of your own, replace these names with the names you've chosen.

Use the following instructions to create the directories and files for a new instance:

1.
Create a new directory for the instance using the following command:

mkdir C:\SQL-NS\NewInstances\MyNewInstance

2.
Copy the files and subdirectories from the Samples\Minimal directory into the NewInstances\MyNewInstance directory using the following command:

xcopy /ev C:\SQL-NS\Samples\Minimal\*.* C:\SQL-NS\NewInstances\MyNewInstance

3.
Rename the Application1 application directory to the name of your application using the following command:

rename C:\SQL-NS\NewInstances\MyNewInstance\Application1 MyNewApplication

After the directory structure and the files for the new instance have been created using the preceding instructions, you must edit some of the files before you can run the build scripts. Use the following instructions to make the required changes:

1.
Open the file C:\SQL-NS\NewInstances\MyNewInstance\Scripts\setenv.cmd in a text editor. The file contains the following text:

call ..\..\..\Common\setenv.cmd
set INSTANCE_NAME=Minimal
set INSTANCE_BASE_DIRECTORY_PATH=%SQLNS_BASE_DIRECTORY_PATH%\Samples\Minimal
set INSTANCE_DATABASE=%INSTANCE_NAME%

2.
In the second line, change Minimal (shown in boldface font) to the name of your instance. In the case of the instance name, MyNewInstance, this line should read:

set INSTANCE_NAME=MyNewInstance

3.
In the third line, replace Samples\Minimal (also shown in boldface font) with the name of the directory that contains your new instance. Note that the Samples part should be replaced with the name of the top-level directory you created for all new instances and the Minimal part should be replaced with the name of this particular instance's subdirectory. Continuing with the example names used so far, this line should be changed to:

set INSTANCE_BASE_DIRECTORY_PATH=%SQLNS_BASE_DIRECTORY_PATH%\NewInstances\MyNewInstance

4.
The default configuration for the new instance will deploy the instance database objects into a database with the same name as the instance. If you want to use a different database name, change the fourth line to set the INSTANCE_DATABASE environment variable to the desired database name. If you want to use the instance name as the database name, no changes are needed to the fourth line.

5.
Save your changes to C:\SQL-NS\NewInstances\MyNewInstance\Scripts\setenv.cmd.

6.
Open the file C:\SQL-NS\NewInstances\MyNewInstance\SQLPermissions.sql in a text editor.

7.
Replace the word Minimal in the USE directive on the first line with the name of your instance database. For example, if the instance database name is MyNewInstance, the first line of the file should read:

USE MyNewInstance

Unless you chose a different name in step 4, the database name is the instance name.

8.
Save your changes to C:\SQL-NS\NewInstances\MyNewInstance\SQLPermissions.sql.

9.
Open the ICF C:\SQL-NS\NewInstances\MyNewInstance\InstanceConfiguration.xml in a text editor.

10.
Change the value of the <InstanceName> element from Minimal to the name of your instance. In the case of the instance name MyNewInstance, the <InstanceName> element should be declared as follows:

<InstanceName>MyNewInstance</InstanceName>

11.
Search for Application1 in the file and replace all occurrences with the name of your application, MyNewApplication. You should find three occurrences.

12.
Change the value of the <DatabaseName> element from Minimal to the name of your instance database. Unless you chose a different name in step 4, the database name is the instance name.

13.
Save your changes to C:\SQL-NS\NewInstances\MyNewInstance\InstanceConfiguration.xml.

14.
Open the ADF C:\SQL-NS\NewInstances\MyNewInstance\MyNewApplication\ApplicationDefinition.xml in a text editor.

15.
Within the <Database> element, you'll need to change the values of the <DatabaseName> and <SchemaName> elements. The configuration scripts assume the instance and application database objects are located in the same database, so you must set the <DatabaseName> element in the ADF to the same value you used for in the <DatabaseName> element in the ICF in step 12.

You can set the <SchemaName> element to any value you choose. It's a common convention to use the application name, but this is not a requirement. You'll need to use the schema name you specify here to qualify the names of the application database objects as you add content to the ADF later on.

Given the example names we've been working with, the <Database> element in the ADF should look like this:

  <Database>
    <DatabaseName>MyNewInstance</DatabaseName>
    <SchemaName>MyNewApplication</SchemaName>
  </Database>

16.
Save your changes to C:\SQL-NS\NewInstances\MyNewInstance\MyNewApplication\ApplicationDefinition.xml.

Building and Running the New Instance

After making the changes described in the previous section, you can use the build scripts to compile the new instance and then start it running. Use the following instructions to do this:

1.
Open a Notification Services Command Prompt and navigate to the Scripts directory of the new instance using the following command:

cd /d C:\SQL-NS\NewInstances\MyNewInstance\Scripts

2.
Run create.cmd to create the new instance. This creates the instance and application database objects.

3.
Run register.cmd to register the new instance.

4.
Run grant_permissions.cmd to grant the required database and filesystem permissions.

5.
Run enable.cmd to enable the instance.

6.
Start the instance using the following command:

net start NS$MyNewInstance

At this point, the new instance is running, although it doesn't do much because it doesn't contain any real code. But you now have a skeleton SQL-NS instance and application to serve as a basis for your development work.

Note

The build scripts described here use the nscontrol command-line tools, but you can also work with the new instance using the graphical tools in Management Studio, as described in Chapter 4.

Adding Code to the New Instance

The ICF for the new instance is in the file C:\SQL-NS\NewInstances\MyNewInstance\InstanceConfiguration.xml. The ADF for the single application, MyNewApplication, is in the file C:\SQL-NS\NewInstances\MyNewInstance\MyNewApplication\ApplicationDefinition.xml. You can edit these files to add new code.

Tip

If you use Management Studio as your XML editor when adding code to the ICF and ADF, you can take advantage of its built-in syntax-checking and element auto-completion features. These can make adding new code much more efficient and less error prone. For details on how to enable these features, see the sidebar, "Support for ADF and ICF Editing in Management Studio," (p. 139) in Chapter 5, "Designing and Prototyping an Application."

As you build the functionality of your SQL-NS instance, I encourage you to use the incremental development approach described in Chapter 5. In this approach, you add code one piece at a time, testing new features in isolation as much as possible. When you add code to your ICF and ADF, you can use the following instructions to update the instance:

1.
Open a Notification Services Command Prompt and navigate to the Scripts directory of the new instance using the following command:

cd /d C:\SQL-NS\NewInstances\MyNewInstance\Scripts

2.
Run disable.cmd to disable the instance.

3.
Stop the new instance's SQL-NS Windows service using the following command:

net stop NS$MyNewInstance

4.
Run update.cmd to compile the changes to the source files and update the instance.

5.
Run enable.cmd to reenable the instance.

6.
Restart the instance using the following command:

net start NS$MyNewInstance

After performing these instructions, the new code should be in effect, and you can begin testing it.


Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]


SYMBOL

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

-application argument
components, enabling 2nd 3rd
-argumentkey parameter 2nd 3rd
-component arguments
components, enabling
-generator argument
disabling generators 2nd
enabling generators
-n argument. [See -name argument]
-name argument. [See -n argument]
-name arguments
instance names, specifying 2nd
-server arguments
registering instances 2nd
-subscriptions arguments
components, enabling
.dll files. [See assemblies]
.NET Framework. [See also Visual Studio .Net 2005]
.NET langauges
custom hosted event provider classes
.NET languages. [See also ASP.NET [NPN]]
[Aaron. Change StockBrokerNMO sample to StockBrokerNMO on all 2nd and 3rd levels]
[brkt-]Action[-brkt] element
developer-defined logic
rule logic 2nd
[brkt-]Action[-brkt] element. [See also [brkt-]ConditionAction[-brkt] element]
[brkt-]Action[-brkt] elements
[brkt-]ActionTimeout[-brkt] element
troubleshooting
[brkt-]Application[-brkt] element
parameter values, troubleshooting
[brkt-]Application[-brkt] elements
[brkt-]ApplicationDefinitionFilePath[-brkt] element 2nd
[brkt-]ApplicationExecutionSettings[-brkt] element
[brkt-]QuantumDuration[-brkt] element 2nd 3rd
event-triggered matching logic
exceed throttles, troubleshooting
StockBrokerNMO
[brkt-]ApplicationExecutionSettings[-brkt] elements
[brkt-]ApplicationName[-brkt] element
[brkt-]Applications[-brkt] element 2nd 3rd
[brkt-]Arguments[-brkt] element
FileSystemWatcherProvider
SQLProvider
[brkt-]AssemblyName[-brkt] elements
custom components, deploying
[brkt-]BaseDirectoryPath[-brkt] element 2nd 3rd
[brkt-]ChronicleQuantumLimit[-brkt] element 2nd
[brkt-]ChronicleRule[-brkt] element
[brkt-]Chronicles[-brkt] element
[brkt-]ClassName[-brkt]element
content formatter interaction
[brkt-]CollationName[-brkt] elements 2nd
[brkt-]ConditionAction[-brkt] element
[brkt-]SqlExpression[-brkt] element 2nd 3rd 4th
conditions
overview 2nd 3rd 4th
types of 2nd 3rd
user-defined rules
user-defined rules. [See also user-defined rules]
implementing 2nd 3rd
internal structure of 2nd
rule processing 2nd 3rd
security
overview 2nd 3rd
user-defined logic rule processing 2nd
versus [brkt-]Action[-brkt] element 2nd 3rd
within [brkt-]EventRule[-brkt] element 2nd
within [brkt-]ScheduleRule[-brkt] element 2nd
[brkt-]ContentFormatter[-brkt] element
[brkt-]ContentFormatter[-brkt] section
[brkt-]ContentFormatter[-brkt] subelements 2nd
[brkt-]Database[-brkt] element 2nd
music store database 2nd
new instances
overview 2nd 3rd
[brkt-]Database[-brkt] elements
in ADF 2nd
in ICF 2nd
[brkt-]DatabaseName[-brkt] element
new instances
[brkt-]DefaultFileGroup[-brkt] element
[brkt-]DeliveryChannel[-brkt] element
SMTP delivery channels, creating
[brkt-]DeliveryChannels[-brkt] element 2nd
[brkt-]DigestGrouping[-brkt] element 2nd 3rd
[brkt-]Distributor[-brkt] declaration
[brkt-]QuantumDuration[-brkt] element 2nd
[brkt-]DistributorLogging[-brkt] element
timeouts, troubleshooting
[brkt-]Distributors[-brkt] element 2nd
[brkt-]EncryptArguments[-brkt] element 2nd 3rd
[brkt-]EventClass[-brkt] element 2nd
declaring event classes 2nd
[brkt-]EventClasses[-brkt] element [do not combine] 2nd
[brkt-]EventClassName[-brkt] element
user-defined logic
[brkt-]eventData[-brkt] element 2nd 3rd
[brkt-]EventRule[-brkt] element
[brkt-]ConditionAction[-brkt] element 2nd
user-defined logic
[brkt-]EventRules[-brkt] element 2nd 3rd 4th
[brkt-]EventThrottle[-brkt] element 2nd
exceed throttles, troubleshooting 2nd
[brkt-]ExpirationAge[-brkt] element
[brkt-]ExpirationAge[-brkt] element, troubleshooting
[brkt-]FailuresBeforeAbort[-brkt] element
[brkt-]FileGroup[-brkt] element 2nd 3rd
[brkt-]FileSpec[-brkt] elements 2nd
[brkt-]Generator[-brkt] elements
[brkt-]GETUTCDATE()[-brkt] SQL function
[brkt-]HostedProvider[-brkt] declaration
SQLProvider
[brkt-]HostedProvider[-brkt] element 2nd
assemblies. [See assemblies]
incorrect schedules, troubleshooting
SQLProvider 2nd 3rd
[brkt-]HostedProvider[-brkt] elements
continuous hosted event providers, declaring 2nd
scheduled hosted event providers, declaring 2nd
[brkt-]IndexSqlSchema[-brkt] element 2nd 3rd
[brkt-]IndexSqlSchema[-brkt] elements
instances, updating
[brkt-]InputName[-brkt] element
user-defined logic 2nd
[brkt-]InputSchema[-brkt] element
user-defined logic 2nd
[brkt-]InstanceName[-brkt] element 2nd
new instances
[brkt-]Interval[-brkt] element
SQLProvider 2nd
[brkt-]LogBeforeDeliveryAttempts[-brkt] element 2nd
[brkt-]LogFile[-brkt] element
[brkt-]LogNotificationText[-brkt] element 2nd
[brkt-]LogStatusInfo[-brkt] element 2nd
[brkt-]NamedFileGroup[-brkt] element
[brkt-]NonHostedProvider[-brkt] element 2nd
declaring 2nd
[brkt-]NonHostedProvider[-brkt]element
[brkt-]NotificationBatchSize[-brkt] element 2nd 3rd
timeouts, troubleshooting
[brkt-]NotificationClass[-brkt] element 2nd 3rd
content formatters, declaring 2nd
[brkt-]NotificationClasses[-brkt] element [do not combine] 2nd
[brkt-]notifications[-brkt] element
XsltFormatter 2nd 3rd
[brkt-]NotificationServicesInstances[-brkt] element 2nd
[brkt-]NotificationThrottle[-brkt] element 2nd
exceed throttles, troubleshooting 2nd
[brkt-]ParameterDefaults[-brkt] element 2nd
[brkt-]ParameterDefaults[-brkt] element in the ADF
[brkt-]ParameterDefaults[-brkt] element in the ICF
[brkt-]Parameters[-brkt] element 2nd
[brkt-]PerformanceQueryInterval[-brkt] element
[brkt-]ProcessEventsInOrder[-brkt] element 2nd
[brkt-]Protocol[-brkt] element
ICF versus ADF
in the ADF
subscriber devices, troubleshooting
[brkt-]Protocols[-brkt] element
application development process
subscriber devices, troubleshooting
[brkt-]Protocols[-brkt] section
[brkt-]Providers[-brkt] elements
[brkt-]QuantumDuration[-brkt] element
event-triggered matching logic 2nd
in [brkt-]ApplicationExecutionSettings[-brkt] element 2nd 3rd
in [brkt-]Distributor[-brkt] element 2nd
[brkt-]RetentionAge[-brkt] element 2nd
manual vaccuuming
[brkt-]Schedule[-brkt] element
SQLProvider 2nd
[brkt-]ScheduledRule[-brkt] element
user-defined logic
[brkt-]ScheduledRules[-brkt] element
[brkt-]ScheduleRule[-brkt] element
[brkt-]ConditionAction[-brkt] element 2nd
[brkt-]Schema[-brkt] element
music store application
[brkt-]SchemaName[-brkt] elements
new instances
[brkt-]SqlExpression[-brkt] element 2nd
within [brkt-]ConditionAction[-brkt] element 2nd 3rd 4th
[brkt-]SqlLogin[-brkt] element
[brkt-]ConditionAction[-brkt] element 2nd 3rd
[brkt-]SqlSchema[-brkt] element
[brkt-]SQLServerSystem[-brkt] element 2nd
[brkt-]SqlServerSystem[-brkt] element
deployment configurations
restoring instances
unable to connect/login errors
[brkt-]SqlStatement[-brkt]
SQLProvider 2nd
[brkt-]SqlStatement[-brkt] element
[brkt-]SqlUser[-brkt] element
[brkt-]ConditionAction[-brkt] element 2nd 3rd
[brkt-]StartTime[-brkt] element
SQLProvider
[brkt-]SubscriptionClass[-brkt] element 2nd 3rd 4th
[brkt-]SubscriptionClasses[-brkt] element
music store application
[brkt-]SubscriptionClasses[-brkt] element [do not combine] 2nd
[brkt-]SubscriptionClassName[-brkt] element
music store application
[brkt-]SubscriptionQuantumLimit[-brkt] element 2nd
[brkt-]SubscriptionThrottle[-brkt] element 2nd
exceed throttles, troubleshooting 2nd
[brkt-]SystemName[-brkt] element
deployment configurations 2nd 3rd
FileSystemWatcherProvider
restoring instances
[brkt-]systemName[-brkt] qualifier
components, enabling 2nd
[brkt-]ThreadPoolSize[-brkt] element 2nd
[brkt-]Vacuum[-brkt] element 2nd
[brkt-]VacuumSchedule[-brkt] element
[brkt-]WorkItemTimeout[-brkt] element
timeouts, troubleshooting
[brkt-]XmlDataFile[-brkt]
[brkt-]xsl
apply-templates/[-brkt] element 2nd
output[-brkt] element
stylesheet[-brkt] element
template[-brkt] element
value-of[-brkt] element
[brkt-]XsltBaseDirectoryPath[-brkt] element


A

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

Abort() method
Event Object APIs 2nd
accounts
deployment security 2nd
accounts. [See user accounts]
Action property
StockBrokerNMO
active event sources 2nd
Add to Database button
event submission 2nd
ADD() method
subscriber objects 2nd
Add() method 2nd
subscriber devices 2nd
AddNewSongByArtistSubscription() method 2nd 3rd
AddNewSongByGenreSubscription() method 2nd
AddSongs program 2nd 3rd
event submission 2nd
AddSubscriber() method
overview 2nd
AddSubscribers.sql
error messages
AddSubscriptions program
expression values, using 2nd 3rd
filter rules and link leaf conditions, creating 2nd 3rd
function values, using 2nd 3rd
overview 2nd
property rules, creating 2nd 3rd
SMO classes, obtaining 2nd 3rd
SongAlerts application, connecting to 2nd
subscription objects, creating 2nd
with compound conditions 2nd 3rd 4th 5th
with simple conditions 2nd 3rd
Visual Studio Solution 2nd 3rd
AddSubscriptions programs
overview
ADF
vacuuming data, configuring 2nd
ADFS
continuous hosted event providers, declaring 2nd
scheduled hosted event providers, declaring 2nd
ADFs
[brkt-]Action[-brkt] elements
[brkt-]ApplicationExecutionSettings[-brkt] elements
[brkt-]Database[-brkt] element
overview 2nd 3rd
[brkt-]Distributors[-brkt] element 2nd
[brkt-]EventClass[-brkt] element 2nd
[brkt-]EventClasses[-brkt] element [do not combine] 2nd
[brkt-]EventRules[-brkt] element 2nd 3rd
[brkt-]Generator[-brkt] elements
[brkt-]NotificationClass[-brkt] element 2nd
[brkt-]NotificationClasses[-brkt] element [do not combine] 2nd
[brkt-]Protocol[-brkt] element
[brkt-]Providers[-brkt] element
[brkt-]QuantumDuration[-brkt] element
[brkt-]SubscriptionClass[-brkt] element 2nd
[brkt-]SubscriptionClasses[-brkt] element [do not combine] 2nd
[brkt-]SystemName[-brkt] element
application development process. [See prototypes (application development process)]
compilation problems, troubleshooting 2nd 3rd 4th
completed ADF examples 2nd 3rd
computer fields
overview 2nd
content formatters, declaring 2nd
custom content formatters, declaring 2nd
delivery protocols
declaring support for 2nd
deployment configurations, specifying 2nd
editing support
Management Studio 2nd
indexes, creating in 2nd 3rd
invalid ADFs
parameters
overview 2nd
skeleton ADFs 2nd
adding code to 2nd
compiling 2nd 3rd 4th
standalone event providers, configuring 2nd
structure of 2nd
troubleshooting
malformed XML
XsltFormatter, declaring 2nd
ADFs (Application Definition Files)
building
overview 2nd
programming to application models 2nd
ADFs (Application Definition Files). [See also schemas]
administration tasks. [See monitoring tasks and maintenance tasks]
administrative SMI (Subscription Management Interface) functions
Administrator accounts. 2nd [See also user accounts]
Administrator privileges
delivery protocols 2nd
aggregation options (application development process) 2nd
ALTER DATABASE statement
analysis reports 2nd
AnalysisServicesProvider
troubleshooting
AND conditions
AND conditions. [See also compound conditions [NPN]]
AndCondition class
APIS
NMO. [See NMO]
APIs
event submission APIs
Event Object APIs 2nd
overview 2nd 3rd
stored procedures for 2nd 3rd 4th
XML Bulkload Event APIs 2nd
Subscriber Management APIs
classes 2nd 3rd
fundamental entities in 2nd
overview 2nd 3rd
SQL-NS assembly
Subscriber Management APIs. [See Subscriber Management APIs]
Windows Data Protection APIs
Application class
StockBrokerNMO
applications, defining 2nd
StockBrokerNMO sample
instances, defining
application configurations 2nd
application database instances, creating (deployment process) 2nd
application database instances, registering (deployment process) 2nd
application database objects
Application Definition Files (ADFs)
programming to application models
Application Definition Files (ADFs). [See ADFs (Application Definition Files)]
application development process
debugging and 2nd
initial design
event-triggered subscriptions 2nd 3rd 4th
event-triggered subscriptions. [See event-triggered subscriptions]
events 2nd 3rd 4th 5th
implementing
notifications. [See notifications]
overview 2nd 3rd
prototypes, building
event-triggered subscriptions 2nd 3rd 4th
event-triggered subscriptions. [See event-triggered subscriptions]
events 2nd 3rd 4th 5th
incremental development 2nd
instances, updating 2nd 3rd 4th 5th 6th 7th 8th
notifications. [See notifications]
overview 2nd
skeleton ADFs 2nd 3rd 4th 5th 6th 7th
testing 2nd 3rd 4th
prototypes, building. [See also music store application]
prototypes, building;
sample application, the. 2nd [See also music store database]
SMIs (Subscription Management Interfaces), building. [See also SMIs (Subscription Management Interfaces) [NPN]]
stages of 2nd
user interfaces
building 2nd
examing 2nd
pages, initializing 2nd
running 2nd 3rd
subscriptions, adding 2nd
subscriptions, displaying 2nd
subscriptions, enabling/disabling/removing 2nd 3rd 4th
technology options 2nd
application directories
Application Event Log
duplicate notification delivery, troubleshooting 2nd
entries 2nd
logging levels 2nd
logging switches 2nd 3rd
overview 2nd
troubleshooting
runtime errors 2nd
timeouts 2nd
Application Event Log [Aaron, consolidate from event log]
Application Event Logs
hosted engines 2nd
application models
overview
programming to
scalability 2nd 3rd
application models. 2nd [See also notification applications] [See also ADFs (Application Definition Files)]
application schemas
database objects
instance metadata 2nd
Registry keys 2nd
application-level performance objects
applications
defining
with NMO 2nd 3rd 4th 5th
enumerating
with NMO 2nd 3rd 4th
submitting events from 2nd 3rd 4th
applications (SQL-NS). [See SQL-NS applications]
Argument class
Condition APIs
argument encryption
event provider security 2nd 3rd 4th
registering instances
argument keys
event provider security 2nd 3rd 4th 5th 6th
registering instances
ArgumentKey property
NMO instance definition
arguments
Hosting APIs 2nd
arithmetic expression terms
ASP.NET
error pages
membership providers. [See membership providers (ASP.NET)] [See membership providers (ASP.NET)]
resources
ASP.NET. [See also Visual Studio [NPN]]
assemblies
continuous hosted event providers
building for 2nd
custom hosted event providers
overview 2nd
scheduled hosted event providers
building for 2nd
signed assemblies
deployment security
assemblies (SMOs)
NMO
overview 2nd 3rd 4th
assemblies. [See also interfaces [NPN]]
asynchronous delivery
distributor optimization
Authentication
hosted engines 2nd
authentication
deployment security 2nd


B

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

BACKUP command 2nd
backups
instances
overview 2nd
batch files
failures in generating, troubleshooting 2nd
setenv.cmd files
batch files. [See also environment variables [NPN]]
between leaf conditions
BetweenLeafCondition property
BodyEncoding delivery channel
Boolean conditions. [See conditions]
boolean leaf conditions
Boolean predicates. [See conditions]
BooleanLeafCondition property
build scripts
deployment security 2nd
new instances, running 2nd
bulk-logged recovery models (database systems) 2nd
business values (notification applications) 2nd
ButtonDisableNewSongByArtistSubscription_Click() helper method 2nd


C

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

C:\SQL-NS\Samples\MusicStore\CreateMusicStore.sql,
troubleshooting
C:\SQL-NS\Samples\StockBroker\InstanceConfiguration.xml,
troubleshooting
cacls utility
filesystem permissions, granting
capacity limitations (input/output systems) 2nd
Chapters directory
chorincle tables
cleaning up 2nd 3rd 4th 5th
chronicle quantum limits. [See also quantum limits]
chronicle rules
event chronicle rules
overview 2nd
chronicle tables
creating indexes over
chronicle update logic
music store application
chronicles
event chronicle rules
overview 2nd
event chronicles
declaring 2nd
overview 2nd
schema qualified names 2nd
testing 2nd
SQLProvider 2nd
subscription chronicles
overview 2nd
user-defined logic 2nd
class hierarchy
NMO classes
classes
custom hosted event providers
overview 2nd
NMO
overview 2nd
Subscriber Management API 2nd
Subscriber Management APIs
classes. [See also assemblies [NPN]]
ClassName property
ContentFormatter class
cleaning up event history
cleanup scripts 2nd 3rd
music store application 2nd
cleanup scripts. [See also maintenace tasks]
cleanup.cmd
music store application 2nd
Click Event handler 2nd 3rd 4th 5th
Click Event handlers
clustered indexes
[brkt-]IndexSqlSchema[-brkt] element
clusters
failover clusters 2nd
in place of SMI web servers
code injection attacks
user-defined logic security 2nd 3rd
codes
adding to new instances 2nd
COM interop
user interfaces, building
command-line tools
development process 2nd
ICFs, compiling 2nd
instance tools 2nd
instances
registering 2nd
instances, enabling 2nd
instances, updating 2nd
Commit() method
Event Object APIs 2nd
MQHostedEventProvider class
Common directory
setenv.cmd file 2nd
compile-time error messages 2nd
compiler. [See SQL-NS compilers]
compilers (SQL)
development process
component-level performance objects
compound conditions
overview 2nd
computed fields
adding 2nd
computed fields (ADFs)
overview 2nd
computed properties. [See also property rules]
computers (for development environments) 2nd
Condition APIs
composition of classes 2nd 3rd
user-defined logic 2nd
Condition APIs (user-defined logic)
Condition class
as base class
conditions
composition of
in Condition APIs 2nd 3rd
compound conditions
overview 2nd
definition of 2nd
expression values, using 2nd 3rd
filter rules
overview 2nd 3rd 4th 5th
function values, using 2nd 3rd
leaf conditions
overview 2nd
link leaf conditions
creating 2nd 3rd
property rules
overview 2nd 3rd
property rules, creating 2nd 3rd
subscriptions, creating
with compound conditions 2nd 3rd 4th 5th
with simple conditions 2nd 3rd
terms
overview 2nd
user-defined logic
overview 2nd 3rd 4th
types of 2nd 3rd
user-defined rules
overview 2nd
Conditions property
Condition APIs
configuration files
NSService.exe.config file. [See NSService.exe.config file]
configuration settings (instances). [See also ICFs (Instance Configuration Files) [NPN]]
Console.ReadKey() method
EngineHost sample application 2nd
constant value terms
ConstantValue class
containment relationships (NMO APIs)
containment relationships (NMO)
content formatters
application development process 2nd
building
advantages of custom content formatters 2nd
overview 2nd
computer fields
adding 2nd
declaring 2nd
declaring in the ADF 2nd
device-specific formatting
device types, specifying 2nd
enabling in music store application 2nd 3rd
overview 2nd 3rd
digest and multicast delivery 2nd
distributors
helper methods 2nd 3rd 4th
IContentFormatter interface
implementing 2nd 3rd 4th
overview 2nd
input and output 2nd
language-specific formatting
enabling in music store application 2nd
overview 2nd 3rd
locale-specific formatting
enabling in music store application
notification bodies
description of
overview 2nd
security 2nd
subscribers/subscriptions
adding for 2nd
testing 2nd
versus distributors 2nd
XsltFormatter. [See XsltFormatter]
content formatters. [See also XsltFormatter [NPN]]
ContentFormatter class
NMO classes
StockBrokerNMO 2nd 3rd
ContentFormatterArgument class
StockBrokerNMO
continuous event providers
continuous event providers. 2nd [See also event providers] [See also custom hosted event providers]
continuous hosted event providers
assemblies
building 2nd
declaring 2nd
Message Queuing 2nd
message queuing event providers 2nd
MQHostedEventProvider class
Commit() method
DoEventSubmission() method 2nd
Initialize() method 2nd
overview 2nd
Receive() method 2nd
Run() method
Terminate() method 2nd
Terminate() method [Aaron-don't connect]
Write() method
overview 2nd
testing 2nd
cost percentages (query execution plans)
costs
subscription matching costs (notification applications) 2nd
CREATE DATABASE statements
filegroups, creating 2nd
CREATE INDEX statements 2nd 3rd
create login command
CREATE LOGIN statement
user-defined logic security
CREATE TABLE statements
music store application 2nd 3rd
Create User button
troubleshooting
CREATE USER command
permissions, granting
Create() method
NMO instance creation 2nd
StockBrokerNMO sample 2nd
CreateSubscriptionObject() helper method
AddSubscriptions program 2nd
credentials. [See also Windows Authentication]
custom components
deployment process
overview 2nd
custom content formatters
building
advantages of 2nd
overview 2nd
declaring in the ADF 2nd
IContentFormatter interface
implementing 2nd 3rd 4th
overview 2nd
testing 2nd
custom delivery protocols
building 2nd
debugging
declaring
in the ICF 2nd
delivery channels
creating for custom delivery protocols 2nd
HTTP-based protocols 2nd
IDeliveryProtocol interface 2nd 3rd 4th
implementing 2nd 3rd 4th 5th
notification classes
supporting in 2nd 3rd
testing 2nd 3rd
restoring system after testing 2nd
custom event providers
overview. 2nd [See also event providers]
troubleshooting 2nd
custom hosted event providers
advantages of 2nd
assemblies
overview 2nd
choosing type of 2nd
classes
overview 2nd
continuous hosted event providers. [See continuous hosted event providers [NPN]]
interfaces
IEventProvider interfaces 2nd
IEventScheduledProvider interfaces 2nd
overview 2nd 3rd
Message Queuing. [See also continuous hosted event providers]
scheduled hosted event providers. [See scheduled hosted event providers [NPN]]
versus built-in event providers


D

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

data files (database systems)
separating from log files 2nd
data flows
efficiency of
normalizing
notification data 2nd
overview 2nd
subscription data 2nd
prefiltering events
overview 2nd 3rd
streamlining 2nd
data layer (SMIs)
connecting to the logic layer 2nd
description of 2nd
Data Protection API (DP API)
hosted engines
Database class
AddSubscriptions program
database objects
application configurations 2nd
changing manually
creating
for membership providers (ASP.NET) 2nd
delivery channels 2nd
delivery protocols 2nd
examing
with nscontrol create 2nd
instance metadata 2nd
instances 2nd
Registry keys 2nd
storing in database systems. [See filegroups]
subscriber data 2nd
subscriber locale tables
timezones tables
database permission
deployment process 2nd
database permissions
music store application (SMIs) 2nd 3rd
user-defined logic security 2nd
database permissions. 2nd [See also permissions]
database roles 2nd
database security 2nd
distributors 2nd 3rd
event provider security 2nd
database systems
deployment security and 2nd
database systems (deployment configurations)
filegroups
creating 2nd 3rd 4th
creating with T-SQL 2nd
file sizes, choosing 2nd
overview 2nd
specifying for database objects 2nd 3rd
overview 2nd
physical database files
data and log files, separating 2nd
file sizes, managing 2nd
hardware considerations 2nd
overview 2nd
tempdb, separating and sizing 2nd
recovery models, setting 2nd
database tables
stored procedures
Databases node (Object Explorer)
databases, exploring 2nd
debugging (rules)
development process and 2nd
generators, reenabling after 2nd
overview 2nd 3rd
preparing for 2nd 3rd
quantum clocks, setting for 2nd 3rd 4th
rule firing
executing 2nd 3rd
preparing for 2nd
semantic errors 2nd
syntax errors 2nd
with scheduled subscription debugging stored procedures 2nd 3rd 4th
declarative programming frameworks (SQL-NS)
default filegroups
delegates
NSInstance.Error event handlers
Delete() method
subscriber devices
subscriber objects 2nd
subscription operations 2nd
DeliverNotification() method
IDeliveryProtocol interface 2nd 3rd 4th 5th 6th 7th
delivery channels 2nd 3rd 4th
BodyEncoding delivery channel
creating for custom delivery protocols 2nd
declaring
in the ICF 2nd
device records
routing notifications to 2nd
SMTP delivery channels
creating 2nd 3rd 4th 5th
subscriber devices, adding 2nd
Subscriber Management APIs
troubleshooitng
subscriber devices 2nd
troubleshooting 2nd 3rd 4th
versus delivery endpoints
versus delivery protocols 2nd 3rd 4th
delivery channels. [See also distributos [NPN]]
delivery endpoints
versus delivery channels
delivery failures 2nd
delivery protocols 2nd
[brkt-]DeliveryChannels[-brkt] element
application development process 2nd
building 2nd
debugging
declaring
in the ICF 2nd
support for in the ADF 2nd
declaring in the ICF 2nd
delivery channels
creating for custom delivery protocols 2nd
delivery failures 2nd
digest/multicast delivery 2nd
distributors 2nd
distributors invocation of
HTTP-based protocols 2nd
IDeliveryProtocol interface 2nd 3rd 4th
implementing 2nd 3rd 4th 5th
Message Queuing
music store application
instances, re-creating 2nd
preparing for 2nd
service account permissions, changing 2nd
SMTP Service, installing 2nd
supporting components, building 2nd
non-starting
troubleshooting 2nd
notification classes
supporting in 2nd 3rd
testing 2nd 3rd
notification distribution views 2nd 3rd 4th
operation of 2nd
overview 2nd
performance optimization 2nd
SMTP delivery protocol. [See also SMTP delivery protocol]
troubleshooting 2nd 3rd 4th
versus delivery channels 2nd 3rd 4th
delivery protocols. [See also distributors [NPN]]
DeliveryChannel class
containment relationships
DeliveryChannelArgument class
StockBrokerNMO sample
deployment configurations
database systems for. [See database systems (deployment configurations)]
high availability configurations 2nd
multiserver configurations
overview 2nd
overview 2nd
scaling out
SQL-NS engines 2nd
scaling the server 2nd
single machine deployment
overview 2nd
SMIs and event providers, separating 2nd
specifying in the ICF and ADFs 2nd
deployment configurations (instances)
deployment machines
SQL-NS, installing on 2nd
deployment machines. [See also deployment configurations]
deployment process
application database instances, creating 2nd
application database instances, registering 2nd
custom components
overview 2nd
enabling instances 2nd
event providers 2nd
overview 2nd
permissions, grating 2nd 3rd 4th
SMIs 2nd
specifying server configurations 2nd
SQL-NS, installing 2nd
starting instances 2nd
deployment security
accountsa and authentication 2nd
database access, controlling 2nd
overview 2nd
security policies
sensitive information, guarding 2nd
throttles, utilizing 2nd 3rd 4th
deployment servers. [See also deployment configurations]
design questions (application development process) 2nd
details (subscribers)
overview. 2nd [See also records (subscribers)]
developer-defined logic
overview
versus user-defined logic 2nd 3rd 4th
developer-defined matching logic
overview
development accounts
setting up 2nd
development enviornments
root directories
NewInstances, creating 2nd
development environements
instances
adding codes to new instances 2nd
creating directories/files for 2nd 3rd
running new instances 2nd
development environments
command-line tools 2nd
computers for 2nd
directory layouts
overview 2nd
environment variables, setting up 2nd 3rd
installing SQL Server components 2nd
local SQL Server configurations
overview
remote SQL Server configurations
service accounts
creating 2nd 3rd
source code
installing 2nd
source files, customizing 2nd
SQL Server 2005 Management Studio 2nd
SQL Server authentications
development accounts 2nd
overview 2nd
setting up 2nd
test accounts 2nd
testing 2nd
user account security 2nd
Visual Studio .NET 2005 2nd
installing 2nd
Windows Server authentications
overview 2nd
setting up 2nd 3rd 4th 5th
development machines versus SQL Servers
development process (applications). [See application development process]
device records
notification batches
device types (content formatters)
device-specific formatting
device types, specifying 2nd
enabling in music store application 2nd 3rd
overview 2nd 3rd
subscribers/subscriptions, adding 2nd
device-specific XSL transforms
directory layouts 2nd
music store application 2nd 3rd
overview
testing 2nd 3rd
DeviceName column
DeviceName fields
overview 2nd
devices (subscribers), manipulating. [See subscriber device operations]
diagnostic reports 2nd
digest delivery
application development process 2nd
content formatting 2nd
delivery protocols 2nd
digest messages
digest parameter (content formatters) 2nd
dilvery channel arguments
encrypting 2nd
directories
creating
for new instances 2nd 3rd
directory structures
new instances
Disable() method
NMOs
disable_generator.cmd
DisableEscaping arguments 2nd 3rd
DisableSubscribers() method
NMOs
DisableSubscriptions() method
NMOs
distribution (notification application phase) 2nd
distribution logs
distributor optimization 2nd 3rd
distribution pipelines
overview 2nd
Distributor class
StockBrokerNMO
distributor logging
timeouts, troubleshooting
versus event logs
distributor logging options
distributor optimization 2nd 3rd
Distributor name
StockBrokerNMO
distributor polling intervals. [See distributor quantums]
distributor quantums 2nd
distributor work items
incomplete distributor work items, troubleshooting 2nd
overview 2nd
distributors
database roles 2nd 3rd
multiple distributors 2nd
optimizing
delivery protocol performance 2nd
distributor logs and 2nd 3rd
distributor quantums and 2nd
overview 2nd
thread pool sizes and 2nd
with multicast delivery 2nd
with notification batch size control 2nd 3rd
overview
troubleshooting
batch files, failure generating 2nd
content formatters/delivery protocols/non-starting 2nd
device names 2nd
duplicate notification delivery 2nd
duplicate notification generation 2nd
generator fall-behind 2nd
incomplete distributor work items 2nd
non-enabled distributors 2nd
non-starting distributors 2nd
notification expirations 2nd
SQL-NS services, non-starting
subscribe device names 2nd
subscriber device configurations 2nd
subscriber devices 2nd
time zones 2nd
timeouts 2nd 3rd
versus content formatters 2nd
distributors. [See also delivery protocols]
DoEventSubmission() method
MQHostedEventProvider class 2nd
domain accounts
creating 2nd
versus local accounts 2nd
DP API (Data Protection API)
hosted engines
DP APIs
DROP TABLE statements
Drop() method
NMOs
duplicate notifications
troubleshooting
delivery of 2nd
generation of 2nd
duration data types (XSD) 2nd
dynamic instance definitions. [See also instances]


E

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

editions
Standard versus Enterprise Editions 2nd
embedded SQL-NS engines. [See Hosting APIs]
Enable() method
NMO instance enabling 2nd
enable.cmd scripts
Enabled column (music store application) 2nd
Enabled property ( SMIs logic layer) 2nd
EnableSubscribers() method
NMOs
EnableSubscriptions() method
NMOs
enabling instances
reasons for
encryption
delivery channel arguments 2nd
encryptions
argument encryption
event provider security 2nd 3rd 4th
EndCollectionTime column (event batch tables)
EngineHost sample application
command-line arguments 2nd
Main() method 2nd 3rd 4th
overview 2nd
source code 2nd 3rd 4th
testing 2nd 3rd
Visual Studio solution 2nd
engines
hosted engines. [See Hosting APIs]
notification applications
deployment of 2nd
scaling out 2nd
Enterprise Managers
Enterprise versus Standard Editions 2nd
entity references. [See escape sequences]
environment variables
deployment security 2nd
setenv.cmd files 2nd
setting up 2nd 3rd
Error events
hosted engines 2nd
EngineHost program 2nd
error messages
monitoring tasks
error messages. [See also troubleshooting]
ErrorCode property
NSException objects
escape sequences
event arrival rates (generator optimization)
event batches
EndCollectionTime column
error handling code
overview 2nd 3rd
quantum ordering 2nd 3rd
StartCollectionTime column
strict event batch ordering 2nd 3rd
submitting 2nd
event chronicle tables
cleaning up 2nd 3rd
event chronicles
duplicate notifications, troubleshooting
troubleshooting 2nd
Event class
Event Object APIs
event class
triggering event classes
user-defined logic
event class declarations 2nd
event classes
declaring 2nd
event data, storing
event handlers
hosted engines
EngineHost program 2nd
Error events 2nd
nsInstance_Error() event handler
EngineHost program 2nd
event history
cleaning up 2nd 3rd
event history, maintaining. [See state mainetenance]
event IDs
troubleshooting 2nd 3rd 4th
Event IDs (event logs)
event logging
hosted engines 2nd
event logs
Application Event Log
entries 2nd
logging levels 2nd
logging switches 2nd 3rd
monitroing tasks 2nd
monitoring tasks 2nd
versus distributor logging
Event Object APIs
error handling codes
overview 2nd
scheduled hosted event providers
event providers
building 2nd
building. [See also custom hosted event providers [NPN]]
built-in event providers
overview 2nd
continuous event providers 2nd
continuous hosted event providers. [See continuous hosted event providers [NPN]]
declaring 2nd
deployment process 2nd
description of 2nd 3rd
event batches
overview 2nd 3rd
event sources
overview 2nd
event submission
from applications 2nd 3rd 4th
with stored procedures 2nd 3rd 4th
event submission APIs
Event Object APIs 2nd
overview 2nd 3rd
stored procedures for 2nd 3rd 4th
XML Bulkload Event APIs 2nd
exeternal event source errors 2nd
FileSystemWatcherProvider
errors
event files/schemas 2nd 3rd
overview 2nd 3rd
permissions 2nd 3rd
runtime behavior of 2nd
testing 2nd 3rd
hosted event providers
overview 2nd
Message Queuing
installing 2nd
Message Queuing. [See also continuous hosted event providers]
music store application
AddSongs program 2nd 3rd
NSEventBatchView
overview 2nd
removing 2nd
runtime errors 2nd
scheduled event providers 2nd
scheduled hosted event providers. [See scheduled hosted event providers [NPN]]
security
argument encryption 2nd 3rd 4th
argument keys 2nd 3rd 4th 5th 6th
database roles 2nd
overview 2nd
separating from SMIs 2nd
SQLProvider
[brkt-]Arguments[-brkt] element
[brkt-]HostedProvider[-brkt] declaration
[brkt-]HostedProvider[-brkt] element 2nd
[brkt-]Interval[-brkt] element 2nd
[brkt-]Schedule[-brkt] element 2nd
[brkt-]StartTime[-brkt] element
chornicles 2nd
events queries 2nd 3rd
overview 2nd
PostQuery arguments
testing 2nd
standalone event providers
advantages of 2nd
building 2nd
configuring in ADFs 2nd
overview 2nd
troubleshooting
incorrect shedules 2nd
non-enabled providers 2nd
SQL-NS non-starts 2nd
unable to start 2nd 3rd
XML event providers
code for 2nd
design of 2nd
LoadXml() method
Main() method 2nd 3rd
overview 2nd
testing 2nd
ValidateArguments() method 2nd
event providers. [See also event submission [NPN]]
event queries
submitting event batches 2nd
event rules
user-defined logic
user-defined logic rule processing
event sources
overview 2nd
troubleshooting 2nd
event states
chronicle rules
overview 2nd
chronicles
declaring 2nd
overview 2nd
schema qualified names 2nd
testing 2nd
overview 2nd
event submission
debugging rules 2nd 3rd
definition of
from applications 2nd 3rd 4th
SQL errors during 2nd
with stored procedures 2nd 3rd 4th
event submission APIs
Event Object APIs 2nd
overview 2nd 3rd
stored procedures for 2nd 3rd 4th
XML Bulkload Event APIs 2nd
event throttles
exceed throttles, troubleshooting 2nd
event throttles (deployment security) 2nd 3rd 4th
event views
user-defined logic rule processing
event-subscription matching 2nd
event-triggered matching logic
match rules, declaring 2nd 3rd
overview 2nd 3rd
testing 2nd 3rd
event-triggered subscriptions
adding 2nd
declaring classes 2nd
music store database 2nd 3rd
overview 2nd
schemas 2nd
EventClass class
NMO classes
StockBrokerNMO 2nd
EventClassName argument
EventCollector class
Event Object APIs 2nd
EventCollectors objects (continuous hosted event providers)
EventField class
NMO classes
StockBrokerNMO 2nd
EventLoader class (XML Bulkload Event API) 2nd
events
application development process
declaring event classes 2nd
declaring event providers 2nd
overview
testing events 2nd
as data 2nd
collection notification application phase 2nd
definition of. [See also notification applications]
prefiltering 2nd 3rd
quantum processing 2nd
schemas for
scoping of data
subscriptions for
testing 2nd
troubleshooting
non-matching subscriptions and events 2nd
events queries
SQLProvider 2nd 3rd
events table
events tables
troubleshooting 2nd
events view
insert triggers
match rules
Exclude action
filter rules
exclude actions
user-defined logic
execution engines. [See SQL-NS execution engines]
expression terms
ExpressionOperator enumeration
AddSubscriptions program
ExpressionValue class
exucution engines (SQL-NS)


F

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

failover clusters 2nd
fall-behind, troubleshooting 2nd
fatal errors
hosted engine event handlers 2nd
EngineHost program 2nd
field value terms
FieldValue class
File Delivery Protocol 2nd
File delivery protocols 2nd
file expansion
filegroups
creating 2nd 3rd 4th
creating with T-SQL 2nd
file sizes, choosing 2nd
overview 2nd
specifying for database objects 2nd 3rd
files
creating
for new instances 2nd 3rd
filesystem permission
deployment process 2nd
filesystem permissions. 2nd [See also permissions]
FileSystemWatcher
troubleshooting
FileSystemWatcherProvider
errors
event files/schemas 2nd 3rd
overview 2nd 3rd
permissions 2nd 3rd
runtime behavior of 2nd
testing 2nd 3rd
filter rules
creating
for AddSubscriptions program 2nd 3rd
overview 2nd 3rd 4th 5th
FilterAction enumeration
Condition APIs
filter rules
FilterRule class
Condition APIs
filters ([brkt-]EventRules[-brkt] element)
FilterStatement class
Condition APIs
finally blocks
scheduled hosted event providers
firewalls
servers outside of
Flush() method
IDeliveryProtocol interface 2nd
foreign keys
FormatContent() method 2nd 3rd
FormatScheduleStart() method 2nd
Formatter property (continuous hosted event providers) 2nd
FormatUSEnglish() method 2nd
FROM clauses
purpose of
full recovery models (database systems) 2nd
fully qualified class names
SubscriptionLoader
fully qualified domain names
specifying
function value terms
FunctionValue class


G

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

garbage collection
generation (notification application phase) 2nd
Generator class
StockBrokerNMO
Generator name
StockBrokerNMO
generators
description of 2nd
disabling 2nd 3rd
enabling 2nd
indexes
creating in ADFs 2nd 3rd
testing effects of 2nd 3rd
opitimizing
scaling up/out the server 2nd
optimizing 2nd
quantum clocks
description of 2nd
falling behind of 2nd 3rd
setting for rule debugging 2nd 3rd 4th
quantum durations
optimizing 2nd 3rd
quantum limits
overview 2nd
quantum processing 2nd
cleanups 2nd
events/subscriptions, selecting 2nd
rule firing 2nd 3rd
stages of 2nd
view materialization 2nd
quantums
definition of
query execution plans 2nd 3rd 4th 5th
query optimization 2nd
troubleshooting
exceeded throttles 2nd
fall-behind 2nd
no events to process 2nd
no subscriptions to process 2nd
non-enabled generators 2nd
non-matching subscriptions and events 2nd
rule timeouts 2nd 3rd
runtime errors 2nd
Genres property (MusicStoreSMIOperations) 2nd
GetNewSongByArtistSubscriptions() function 2nd
GetNewSongByGenreSubscriptions() function 2nd
GetSelectedNewSongByArtistSubscription() helper method 2nd
GETUTCDATE() SQL function
Global Assembly Cache (GAC)
strong names
GO directives
granularity of scheduled subscriptions 2nd
graphical SQL-NS tools. [See GUI SQL-NS tools]
growth increments (physical database files)
GUI SQL-NS tools
ICFs, compiling 2nd
instances 2nd


H

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

hash matches
query execution plans
Hashtable class
helper methods
content formatter class 2nd 3rd 4th
helper operation (MusicStoreSMIOperations)
Genres property 2nd
GetNewSongByArtistSubscriptions() function 2nd
GetNewSongByGenreSubscriptions() function 2nd
overview
SubscriberExists() function 2nd
Subscribers property 2nd
TimeZones property 2nd
high availability configurations 2nd
hosted event providers
building. [See custom hosted event providers [NPN]]
overview 2nd
hosted event providers. [See also event providers [NPN]]
Hosting APIs
Application Event Logs 2nd
creating hosted engines
overview 2nd
definition of 2nd
EngineHost program
Error events 2nd
EngineHost sample application
command-line arguments 2nd
Main() method 2nd 3rd 4th
overview 2nd
source code 2nd 3rd 4th
testing 2nd 3rd
Visual Studio solution 2nd
Error events
EngineHost program 2nd
overview 2nd
IsRunning property
purpose of 2nd
security/authentication
overview 2nd
starting hosted engines
multiple engine starts 2nd 3rd 4th
overview 2nd
StopInstance() method
overview 2nd
using
overview 2nd
Windows service and 2nd
HTTP-based protocols 2nd
HttpExtension class


I

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

ICalendar syntax 2nd
ICF
[brkt-]Protocol[-brkt] element
delivery protocols
declaring 2nd
ICFs
[brkt-]ApplicationDefinitionFilePath[-brkt] element 2nd
[brkt-]ApplicationName[-brkt] element
[brkt-]Applications[-brkt] element 2nd 3rd
[brkt-]BaseDirectoryPath[-brkt] element 2nd 3rd
[brkt-]Database[-brkt] element 2nd
[brkt-]DeliveryChannels[-brkt] element 2nd
[brkt-]ParameterDefaults[-brkt] element 2nd
[brkt-]Parameters[-brkt] element 2nd
compilation problems, troubleshooting 2nd 3rd 4th
compiling
overview 2nd
with command-line tools 2nd
with GUI SQL-NS tools 2nd
delivery channels/protocols
declaring 2nd
deployment configuratioins, specifying
deployment configurations, specifying
editing support
Management Studio 2nd
parameters 2nd
root element
[brkt-]InstanceName[-brkt] and [brkt-]SQLServerSystem[-brkt] elements 2nd
overview
root elements
troubleshooting
malformed XML
ICFs (Instance Configuration Files) 2nd
configuring
definition of
elements of
overview 2nd
permissions
Registry Keys
Windows Service
IContentFormatter interface
implementing 2nd 3rd 4th
overview 2nd
IDeliveryProtocol interface
overview 2nd 3rd 4th
IEventProvider interfaces
overview 2nd
IEventProvider. [See also continuous hosted event providers [NPN]]
IEventScheduledProvider interfaces
overview 2nd
IHttpProtocolProvider
impersonation (user-defined logic security) 2nd 3rd
Include action
filter rules
include actions
user-defined logic
incremental development (application development process) 2nd
skeleton ADFs 2nd
adding code to 2nd
compiling 2nd 3rd 4th
index operator ([])
index seeks
indexes
creating in ADFs 2nd 3rd
query execution plans 2nd 3rd 4th 5th
query optimization 2nd
testing effects of 2nd 3rd
IndexOutOfRangeException
informational messages
monitoring tasks
initial design (application development process)
event-triggered matching logic
match rules, declaring 2nd 3rd
overview 2nd 3rd
testing 2nd 3rd
event-triggered subscriptions
adding 2nd
declaring classes 2nd
overview 2nd
schemas 2nd
events
declaring event classes 2nd
declaring event providers 2nd
overview
testing events 2nd
implementing
notification classes
compiling 2nd
notifications
aggregation options 2nd
delivery protocols 2nd
formatting 2nd
notification schemas 2nd
overview 2nd
overview 2nd 3rd
initial file sizes (physical database files)
Initialize() method
IDeliveryProtocol interface 2nd
IEventProvider Interface 2nd 3rd
MQHostedEventProvider class 2nd
ScheduledHostedEventProvider class 2nd 3rd
input data (property rules)
input systems (notification applications)
capacity limitations 2nd
input types (property rules)
InputType property
property rules
insert triggers
installations
prerequisites/system requirements
SQL Server components 2nd
installations (SQL-NS)
on deployment machines 2nd
Instance class
containment relationships
Instance class (NMO). [See NMO]
instance directories
Instance Properties dialog box (Management Studio) 2nd
instance-level performance objects
Instance.Create() method
NMO instance creation 2nd
instance.Create() method
InstanceDatabaseOptions class
NMO instance definition
instances
backing up
overview 2nd
changing manually
cleaning up 2nd
codes
adding to new instances 2nd
command-line tools 2nd
compiling 2nd
creating
with NMO 2nd 3rd 4th
creating directories/files for 2nd 3rd
creating. 2nd 3rd [See also ICFs (Instance Configuration Files)]
data loss/corruption
database objects 2nd
application configurations 2nd
delivery channels 2nd
delivery protocols 2nd
examing with nscontrol create 2nd
subscriber data 2nd
subscriber locale tables
timezones tables
defining
with NMO 2nd 3rd 4th 5th 6th
deleting
with NMO 2nd
deployed instances, enabling 2nd
deployed instances, starting 2nd
deploying
on multiple servers 2nd
disabling
with NMO 2nd
enabling
reasons for
with Management Studio 2nd
with NMO 2nd
with nscontrol enable command 2nd
enumerating
with NMO 2nd
GUI tools 2nd
metadata 2nd 3rd
multiple side-by-side 2nd
overview 2nd 3rd 4th
permission, granting
filesystem permissions 2nd
permissions, granting
database permissions 2nd
reflecting. [See reflections (NMO)]
registering
with command-line tools 2nd
with Management Studio 2nd
with NMO 2nd
registration, changing
Registry keys 2nd
removing 2nd
restoring
overview 2nd
running new instances 2nd
SQL errors, troubleshooting
overview 2nd
permission denied errors 2nd
unable to connect/log in errors 2nd
SQL-NS instances versus SQL Server instances
starting 2nd
Subscriber Management APIs 2nd
tools for
overview 2nd
unregistering
with NMO 2nd
updating
overview 2nd 3rd 4th 5th 6th 7th 8th
performing updates 2nd
post-update steps 2nd
preparing for 2nd
using Management Studio 2nd
using nscontrol update 2nd
with NMO 2nd
IntelliSense
interfaces
custom hosted event providers
IEventProvider interfaces 2nd
IEventScheduledProvider interfaces 2nd
overview 2nd 3rd
interfaces. [See also classes [NPN]]
intermediate tables
joins with 2nd
Internet Information Server (IIS)
IOTimeout
continuous hosted event providers
is-null leaf conditions
IScheduledEventProvider. [See also scheduled hosted event providers [NPN]]
IsNullLeafCondition property
IsRunning property
hosted engines 2nd


J

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

join clauses
user-defined logic 2nd 3rd
Join() methods
continuous hosted event providers
JoinClause
filter rules/link leaf conditions, creating
joins
filter rules/link leaf conditions, creating
joins. [See SQL joins]


L

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

language-specific formatting
overview 2nd 3rd
lastRunTime variable
scheduled hosted event providers
leaf conditions
filter rules and
overview 2nd 3rd 4th 5th
overview 2nd
subscriptions, creating
with simple leaf conditions 2nd 3rd
licensing
edition differences
link chains (join clauses)
link leaf conditions
creating
for AddSubscriptions program 2nd 3rd
link operators
description of 2nd
LinkLeafCondition property
LinkLeafOperator enumeration
LoadXml() method
XML event providers
local accounts
creating 2nd
versus domain accounts 2nd
local SQL Server configurations
development environments
locale codes 2nd
Locale fields 2nd
locale-specific formatting
enabling in music store application 2nd 3rd
overview 2nd 3rd
subscribers/subscriptions, adding 2nd
locale-specific XSL transforms
directory layouts 2nd
music store application 2nd 3rd
overview
testing 2nd 3rd
lock constructs
log files (database systems)
separating from data files 2nd
log switches 2nd 3rd
hosted engines
logging (distributors). [See distributor logs]
logging levels
hosted engines 2nd
setting 2nd
logic layer (SMIs)
codes 2nd
connecting to the data layer 2nd
description of 2nd
implementing 2nd
MusicStoreSMIOperations class
Genres property 2nd
GetNewSongByArtistSubscriptions() function 2nd
GetNewSongByGenreSubscriptions() function 2nd
overview
SubscriberExists() function 2nd
Subscribers property 2nd
TimeZones property 2nd
operations 2nd
purpose of 2nd
subscriber device operations
adding subscriber devices 2nd
overview
removing subscriber devices 2nd 3rd
subscriber operations
overview 2nd
subscriber records
modifying 2nd
overview 2nd
removing 2nd 3rd
Subscription class
Add() method 2nd
AddNewSongByArtistSubscription() method 2nd 3rd
AddNewSongByGenreSubscription() method 2nd
Delete() method 2nd
FormatScheduleStart() method 2nd
index operator ([])
overview 2nd
ScheduleRecurrence properties 2nd
ScheduleStart properties 2nd
TimeZoneId property
ToString() method
Update() method 2nd
logic layer (SMIs). [See also Subscriber Management APIs]
Login property
StockBrokerNMO sample
logins
creating
LoginSecure property
StockBrokerNMO sample
low privilege accounts
user-defined logic security
low privileged accounts
creating
hosted engines


M

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

Main() method
EngineHost sample application 2nd
XML event providers 2nd 3rd
MainForm_Load() method
reflections (NMO) 2nd
maintenance tasks
components, enabling/disabling
overview 2nd
state changes, applying 2nd 3rd
with Management Studio 2nd
with nscontrol 2nd 3rd
definition of
overview 2nd
maintenance techniques
cleanups
chronicle tables 2nd 3rd 4th 5th
manual vacuuming 2nd
overview 2nd
updating instances
overview 2nd 3rd 4th
performing updates 2nd
post-update steps 2nd
preparing for 2nd
malicious users (deployment configurations). [See deployment security]
managed codes (Subscriber Management APIs)
overview 2nd
Management Studio
ADFs
editing support 2nd
ADFs, bringing up
argument keys, specifying
components, enabling/disabling 2nd
development process 2nd
ICFs
editing support 2nd
ICFs, compiling 2nd
instance tools 2nd
instances
registering 2nd
instances, cleaning up
instances, enabling 2nd
instances, starting 2nd
list of applications, viewing
new instances 2nd
NMO. [See NMO]
Update Instance command 2nd 3rd 4th 5th
troubleshooting 2nd 3rd 4th
ManageSubscriptions class. [See also user interfaces [NPN]]
ManageSubscriptions.aspx web form
manual vacuuming 2nd
match rule
developer-defined match rules
match rules
declaring 2nd 3rd
description of
DeviceName/Locale values, passing 2nd
query execution plans 2nd
troubleshooitng
subscriber device name insertion 2nd
troubleshooting
non-matching subscriptions and events
match rules. [See also matching logic [NPN]]
matches view
user-defined logic
matching
with SQL joins 2nd 3rd
matching (subscriptions) 2nd
matching components
as SQL-NS application core
matching criteria. [See also user-defined matching logic]
matching logic
[brkt-]EventRules[-brkt] element 2nd 3rd
developer-defined matching logic
overview 2nd
developer-defined versus user-defined 2nd
evaluating with queries 2nd 3rd 4th 5th
event-triggered subscriptions
overview 2nd 3rd
event-triggered subscriptions. [See match rules]
user-defined matching logic
overview 2nd
user-defined matching logic. [See user-defined matching logic]
user-defined versus developer-defined logic 2nd
membership providers (ASP.NET)
database objects, creating for 2nd
Message objects
continuous hosted event providers
Message Queuing
delivery protocols
event providers. 2nd [See also continuous hosted event providers]
installing 2nd
overview 2nd
MessageQueue objects (continuous hosted event providers)
MessageQueueException objects
continuous hosted event providers
messages (event logs). [See event logs]
metadata
instances
metadata (instances) 2nd
Microsoft SQL Server 2005. [See SQL Server 2005]
Microsoft.Sql.Server.NotificationServices.dll, [See SQL-NS assembly]
Microsoft.SqlServer.ConnectionInfo.dll assemblies
Microsoft.SqlServer.ConnectionInfo.dll assembly
AddSubscriptions program
Microsoft.SqlServer.Management.Common assemblies
Microsoft.SqlServer.Management.Common namespace
AddSubscriptions program
Microsoft.SqlServer.Management.Nmo assemblies
Microsoft.SqlServer.Management.Smo assemblies
Microsoft.SqlServer.Management.Smo namespace
AddSubscriptions program
Microsoft.SqlServer.NotificationServices namespace 2nd
interfaces. [See also interfaces]
Microsoft.SqlServer.NotificationServices. [See SQL-NS namespace]
Microsoft.SqlServer.NotificationServices.dll assemblies 2nd
Microsoft.SqlServer.NotificationServices.Rules namespace
Condition APIs
Microsoft.SqlServer.NotificationServices.Rules.dll assembly
Condition APIs
Microsoft.SqlServer.Smo.dll assemblies
Microsoft.SqlServer.Smo.dll assembly
AddSubscriptions program
Minimal sample
description of
monitoring tasks
Application Event log
entries 2nd
logging levels 2nd
logging switches 2nd 3rd
overview 2nd
definition of
overview 2nd
performance counters
overview 2nd 3rd 4th
reporting stored procedures
analysis reports 2nd
diagnostic reports 2nd
overview 2nd 3rd
snapshot reports 2nd
MQDeliveryProtocol 2nd
MQDeliveryProtocol.cs file 2nd 3rd 4th 5th
MQHostedEventProvider class
Commit() method
DoEventSubmission() method 2nd
Initialize() method 2nd
overview 2nd
Receive() method 2nd
Run() method
Terminate() method 2nd
Terminate() method [Aaron-don't connect]
Write() method
MQNotification classes 2nd
MQNotification objects 2nd
msdb system databases
multicast delivery
application development process 2nd
content formatting 2nd
delivery protocols 2nd
distributor optimization 2nd
Enterprise Edition support
multimachine deployment configurations
overview 2nd
multiserver deployment configurations
overview 2nd
music store appliaction
AddSubscriptions program
compound conditions 2nd 3rd 4th 5th
expression values, using 2nd 3rd
filter rules and link leaf conditions, creating 2nd 3rd
function values, using 2nd 3rd
overview 2nd 3rd
property rules, creating 2nd 3rd
simple conditions 2nd 3rd
SMO classes, obtaining 2nd 3rd
SongAlerts application, connecting to 2nd
subscription objects, creating 2nd
Visual Studio Solution 2nd 3rd
user-defined logic
[brkt-]ConditionAction[-brkt] element, implementing 2nd 3rd
input, creating 2nd 3rd
overview 2nd 3rd 4th
security 2nd 3rd
subscription classes, defining for 2nd
testing the application 2nd 3rd 4th 5th
music store application
[brkt-]Arguments[-brkt] element
SQLProvider
[brkt-]HostedProvider[-brkt] declaration
SQLProvider
[brkt-]HostedProvider[-brkt] element
SQLProvider 2nd
[brkt-]Interval[-brkt] element
SQLProvider 2nd
[brkt-]Schedule[-brkt] element
SQLProvider 2nd
[brkt-]StartTime[-brkt] element
SQLProvider
Add to Database button
event submission 2nd
AddSongs program 2nd 3rd
event submission 2nd
chronicles
SQLProvider 2nd
cleanup scripts 2nd
content formatters, preparing for
computed fields, adding 2nd
device and locale storage 2nd 3rd
event submission
extra event providers, removing 2nd
instances, creating 2nd
overview 2nd
subscribers/subscriptions, adding 2nd
custom delivery protocols. [See custom delivery protocols [NPN]]
delivery channels
subscribe devices, adding 2nd
delivery protocols
instances, re-creating 2nd
preparing for 2nd
service account permissions, changing 2nd
SMTP Service, installing 2nd
supporting components, building 2nd
FileSystemWatcherProvider 2nd 3rd
instances
compiling 2nd
locale-specific XSL transforms 2nd 3rd
testing 2nd 3rd
PostQuery arguments
SQLProvider
scheduled subscriptions
overview 2nd
testing 2nd 3rd
SMTP delivery protocol
creating 2nd 3rd 4th 5th
declaring support in the Notification classes 2nd 3rd
testing 2nd 3rd
utilizing 2nd
SQLProvider
[brkt-]Arguments[-brkt] element
[brkt-]HostedProvider[-brkt] declaration
[brkt-]HostedProvider[-brkt] element 2nd
[brkt-]Interval[-brkt] element 2nd
[brkt-]Schedule[-brkt] element 2nd
[brkt-]StartTime[-brkt] element
chornicles 2nd
events queries 2nd 3rd
overview 2nd
PostQuery arguments
testing 2nd
testing 2nd 3rd
user interfaces
building 2nd
examing 2nd
pages, initializing 2nd
running 2nd 3rd
subscriptions, adding 2nd
subscriptions, displaying 2nd
subscriptions, enabling/disabling/removing 2nd 3rd 4th
technology options 2nd
user interfaces, building. [See also SMIs (Subscription Management Interfaces) [NPN]]
music store applications
event and scheduled rules interaction 2nd
scheduled subscription class
overview
scheduled rules 2nd 3rd
schema for subscriptions by genre 2nd
state mainetenance
event chronicle rules 2nd
event chronicles 2nd 3rd 4th 5th 6th
overview 2nd
schema qualified names 2nd
subscription chronicles 2nd
subscription states 2nd
music store catalog. [See music store database [NPN]]
music store database
creating 2nd 3rd
purpose of designing 2nd
schema of 2nd
music store database. [See also application development process [NPN]]
MusicStoreSMIOperations class (SMI logic layer)
Genres property 2nd
GetNewSongByArtistSubscriptions() function 2nd
GetNewSongByGenreSubscriptions() function 2nd
overview
SubscriberExists() function 2nd
Subscribers property 2nd
TimeZones property 2nd


N

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

namespace aliases
MainForm.cs
namespaces
NMO
overview 2nd 3rd 4th
schemas as
nested loops joins
net start command
instances, starting
launching of new instances 2nd
SQL-NS engine, starting
NetBIOS names
specifying
New Notification Services Instance command
ICFs, compiling 2nd
troubleshooting 2nd 3rd 4th
newsgroups
SQL-NS public newsgroup
NMO
applications
defining 2nd 3rd 4th 5th
enumerating 2nd 3rd 4th
classes
overview 2nd
compared to nscontrol/Management Studio 2nd
instances
creating 2nd 3rd 4th
defining 2nd 3rd 4th 5th 6th
deleting 2nd
disabling 2nd
enabling 2nd
enumerating 2nd
obtaining objects
registering 2nd
unregistering 2nd
updating 2nd
namespaces
overview 2nd 3rd 4th
overview 2nd
reflections
applications, enumerating 2nd 3rd 4th
instances, enumerating 2nd
overview 2nd
preparing for 2nd
subscription classes, enumerating 2nd 3rd 4th
SubscriptionLoader code 2nd 3rd
sample application
overview 2nd 3rd 4th
subscription classes
enumerating 2nd 3rd 4th
NMO namespaces
NMOs
StockBrokerNMO sample
testing 2nd
non-hosted event providers. [See standalone event providers]
nonclustered indexes
nonhosted event providers
application development process
nonhosted event providers. [See also event providers]
NonHostedEventProvider class
StockBrokerNMO
normalization (data flows)
notification data 2nd
overview 2nd
subscription data 2nd
NOT conditions
NOT conditions. [See also compound conditions [NPN]]
NotCondition class
notification applications
application models
overivew 2nd
challenges in building
overview 2nd
reliability 2nd
rich application features 2nd
scalability 2nd 3rd
security and privacy 2nd
declarative programming frameworks (SQL-NS)
engine execution (SQL-NS)
events as data 2nd
execution engines (SQL-NS)
framework extensibility (SQL-NS)
new features (SQL-NS) 2nd
processing phases of 2nd
distribution 2nd
event collection 2nd
generation 2nd
subscription management 2nd
publish-subscribe models 2nd
components of 2nd
removing 2nd
request-response model 2nd
rich application features
SQL-NS
running
example of 2nd 3rd
scalability 2nd 3rd
speed of development (SQL-NS)
SQL-NS benefits 2nd 3rd 4th
subscriptions as data 2nd
notification applications. [See also application models]
notification applicatoins
stock application example, extending 2nd
notification batch files
failure in generating, troubleshooting 2nd
notification batch size settings
Enterprise Edition support
notification batch sizes, controlling 2nd 3rd
notification batches
overview 2nd
notification bodies
description of
Notification classes
SMTP delivery protocols, declaring support for 2nd 3rd
notification classes
compiling
application development process 2nd
delivery protocols
supporting in 2nd 3rd
notification data
normalizing 2nd
notification distribution views
delivery protocols 2nd 3rd 4th
notification expirations
delivery protocols 2nd
notification expirations, troubleshooting 2nd
notification generators. [See generators]
Notification Management Objects (NMO) API
notification schemas 2nd
application development process 2nd
Notification Services
parameters (source files) 2nd
notification services
installing 2nd
Notification Services Management Objects. [See NMO]
Notification Services. [See also SQL Server 2005 Notification Services]
notification throttles
exceed throttles, troubleshooting 2nd
notification throttles (deployment security) 2nd 3rd 4th
NotificationClass class
NMO classes
StockBrokerNMO
NotificationClassProtocol class
StockBrokerNMO
NotificationHeaders objects 2nd
notifications
application development process
aggregation options 2nd
delivery protocols 2nd
formatting 2nd
notification schemas 2nd
overview 2nd
delivery of. 2nd [See also delivery protocols]
production of
routing
to delivery channels 2nd
schemas for
notifications table
Notifications view
subscriber device names, troubleshooting 2nd
notifications view
[brkt-]ConditionAction[-brkt] element
row insertion
match rules
rewriting
user-defined logic
NotificationServices class
NMO classes
NotificationServices event source
duplicate notification delivery, troubleshooting
timeouts, troubleshooting 2nd
NotificationServices event sources 2nd
timeouts, troubleshooting
troubleshooting
NotificationStatus objects 2nd 3rd
NotificationStatusCallback delegates
NS[brkt-]EventClass[-brkt]EventBatches tables
NS[brkt-]SubscriptionClass[-brkt]Schedules table [Aaron, consider deleting this entry]
instances, updating
NS[brkt-]SubscriptionClass[-brkt]SubscriptionsOld 2nd
NS[brkt-]SubscriptionClass[-brkt]View 2nd
NSApplication class
AddSubscriptions program 2nd
NSApplication class (Subscriber Management API) 2nd
NSApplication constructor
NSApplication constructors 2nd
NSApplicationNames table
nscontrol
instance registrations, changing
NMO. [See NMO]
nscontrol commands
setenv.cmd files
troubleshooting 2nd 3rd 4th
nscontrol create
instance database objects, examining 2nd
nscontrol create command
ICFs, compiling 2nd
troubleshooting 2nd 3rd 4th
nscontrol delete command
instances, cleaning up 2nd
metadata deletion 2nd
troubleshooting 2nd 3rd 4th
nscontrol disable command
nscontrol disable command
individual components 2nd 3rd
nscontrol enable
nscontrol enable command
individual components 2nd 3rd
instances, enabling 2nd
troubleshooting 2nd 3rd 4th
nscontrol export command
nscontrol register command 2nd
instances, registering 2nd
nscontrol register command. [See performance counters]
nscontrol repair command
metadata, rebuilding
nscontrol unregister command
instances, cleaning up 2nd
nscontrol update
instances, updating 2nd
nscontrol update command 2nd 3rd 4th 5th
troubleshooting 2nd 3rd 4th
nscontrol.exe
instances 2nd
NSDiagnosticEventClass stored procedure
NSDistributor database roles
NSEventBatchView
NSEventBeginBatch stored procedure (SQL)
NSEventBeginBatchSongAdded stored procedure
NSEventFlushBatch[brkt-]EventClass[-brkt] stored procedure
NSEventFlushBatchSongAdded stored procedure
NSEventProvider role 2nd
database security
NSEventSubmitBatch[brkt-]EventClass[-brkt] stored procedure
NSEventSubmitBatchSongAdded
NSEventWrite[brkt-]EventClass[-brkt] stored procedure
NSEventWriteSongAdded stored procedure
NSException class (Subscriber Management API) 2nd
NSException objects
ErrorCode property
overview 2nd
NSExecuteRuleFiring
with query execution plans on
NSExecuteRuleFiring stored procedure 2nd 3rd 4th 5th
NSInstance class
AddSubscriptions program 2nd
hosted engines
versus NMO
NSInstance class (Subscriber Management API)
NSInstance class. [See also Hosting APIs [NPN]]
NSInstance constructor
hosted engines, creating 2nd
NSInstance constructors 2nd
nsInstance_Error() event handler
EngineHost program 2nd
NSPrepareRuleFiring stored procedure 2nd 3rd 4th
NSRunService database role
permissions, granting
NSRunService role
database security
NSScheduledSubscriptionDetails stored procedure
debugging with 2nd
time zones, troubleshooting
NSScheduledSubscriptionDetails stored procedure. [See also scheduled subscription debugging stored procedures]
NSScheduledSubscriptionList stored procedure
debugging with 2nd
time zones, troubleshooting
NSScheduledSubscriptionList stored procedure. [See also scheduled subscription debugging stored procedures]
NSService.exe.config file
logging levels
hosted engines 2nd
NSService.exe.config files 2nd
troubleshooting event providers 2nd 3rd
NSSetQuantumClock stored procedure 2nd
versus NSSetQuantumClockDate stored procedure 2nd
NSSetQuantumClockDate stored procedure 2nd
versus NSSetQuantumClock stored procedure 2nd
NSSubscriberAdmin role
database permissions (subscription management)
NSSubscriberDevices table
troubleshooting 2nd
NSSubscribers table
NSVacuum stored procedure
manual vacuuming 2nd


O

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

Object Explorer
Databases node
View node
Object Explorer. [See also Management Studio [NPN]]
operations
logic layer
overview 2nd
OR conditions
OR conditions. [See also compound conditions [NPN]]
OrCondition class
ordering models
user-defined logic rule processing
output systems (notification applications)
capacity limitations 2nd
overheads (generator checks)


P

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

Page Load() handler 2nd
pages (music store application), initializing 2nd
parallelism, increasing
parameters
missing values, troubleshooting 2nd
parameters (source files)
overview 2nd
parameters grids (New Notification Service Instance dialog box)
Parent property
NMO classes
parent-child relationships (NMO) 2nd 3rd
pass-through XSL transforms
passive event sources 2nd
Password property
StockBrokerNMO sample
passwords
deployment security 2nd
registering instances
security
passwords in code
perfmon tool 2nd
performance counters
ICFs (Instance Configuration Files)
overview 2nd 3rd 4th
performance monitoring. [See performance counters]
performance objects 2nd
performance tuning
measurements, importance of
overview 2nd
permissions
database permissions
music store application (SMIs) 2nd 3rd
deployment process 2nd 3rd 4th
FileSystemWatcherProvider 2nd 3rd
granting
database permissions 2nd
filesystem permissions 2nd
ICFs (Instance Configuration Files)
setting up 2nd 3rd 4th 5th 6th
permissions. 2nd [See also Administrator accounts]
physical database files
data and log files, separating 2nd 3rd 4th
file sizes, managing 2nd
hardware considerations 2nd
overview 2nd
polling intervals
polymorphism (condition composition)
PopulateSubscriptions() method 2nd
PostQuery arguments
SQLProvider
prefiltering events 2nd 3rd
prerequisites
SQL-NS
PRIMARY filegroup
privacy (notification applications)
overview 2nd
privileges
hosted engines
processing latencies 2nd
programmatic management (SMOs)
programming
to application models 2nd
property rules
creating
for AddSubscriptions program 2nd 3rd
overview 2nd 3rd
result values
statement orders in
PropertyRule class
Condition APIs
PropertyRule statement
property rules
PropertyStatement class
Condition APIs
protocol fields
overview 2nd
SMTP delivery protocol 2nd
ProtocolDefinition class
NMO instance definition
prototypes (application development process)
building
overview
description of 2nd
prototypes (application development process), building
event-triggered matching logic
match rules, declaring 2nd 3rd
overview 2nd 3rd
testing 2nd 3rd
event-triggered subscriptions
adding 2nd
declaring classes 2nd
overview 2nd
schemas 2nd
events
declaring event classes 2nd
declaring event providers 2nd
overview
testing events 2nd
incremental development 2nd
skeleton ADFs 2nd 3rd 4th 5th 6th 7th
instances, updating 2nd 3rd 4th 5th 6th 7th 8th
notification classes
compiling 2nd
notifications
aggregation options 2nd
delivery protocols 2nd
formatting 2nd
notification schemas 2nd
overview 2nd
overview
testing 2nd 3rd 4th
prototypes (application development process), building. [See also music store application]
prototypes (application development process), building;
ProviderName columns
providers (event). [See event providers]
pub-sub model. [See publish-subscribe models]
public key tokens
deployment security
publish-subscribe models
overview 2nd
publish-subscribe models. [See also notification applications]


Q

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

quantum clocks
fall behind, troubleshooting 2nd
falling behind of 2nd 3rd
overview 2nd
setting for rule debugging 2nd 3rd 4th
quantum durations
optimizing 2nd 3rd
quantum end times
quantum IDs, obtaining 2nd
quantum limits
fall behind, troubleshooting
overview 2nd
versus throttles
quantum ordering 2nd 3rd
quantum ordering models
user-defined logic rule processing
quantum processing 2nd
cleanups 2nd
events/subscriptions, selecting 2nd
rule firing 2nd 3rd
stages of 2nd
view materialization 2nd
quantum processing model
user-defined logic rule processing 2nd
quantum start times
QuantumDuration property
StockBrokerNMO
quantums
definition of
distributor quantums 2nd
user-defined logic rule processing 2nd
vaccuuming and
quantums. [See also quantum clocks [NPN]]
queries
filter rules. [See filter rules]
matching logic evaluation 2nd 3rd 4th 5th
Query Analyzers
query execution plans 2nd 3rd 4th 5th
indexes
creating in ADFs 2nd 3rd
testing effects of 2nd 3rd
query optimization
indexes 2nd
query plans. [See query execution plans]


R

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

RAID disk systems
Receive() method
MQHostedEventProvider class 2nd
records (subscribers)
modifying 2nd
overview 2nd
removing 2nd 3rd 4th 5th
recovery models (database systems) 2nd
reflections
SMO classes
reflections (NMO)
applications, enumerating 2nd 3rd 4th
instances, enumerating 2nd
overview 2nd
preparing for 2nd
subscription classes, enumerating 2nd 3rd 4th
SubscriptionLoader code 2nd 3rd
Register Instance dialog box (Management Studio)
register with argument key.cmd
versus register.cmd
register.cmd
versus register with argument key.cmd
registering
instances
with command-line tools 2nd
with Management Studio 2nd
RegisterLocal() methods
NMO instance registration 2nd
RegisterLocal() overloads
NMO instance registration 2nd 3rd
registery keys
hosted engines 2nd
Registry keys
ICFs (Instance Configuration Files)
reliability (notification applications)
overview 2nd
remote SQL Server configurations
development environments
RemoveSubscriber() method
subscriber objects 2nd
reporting stored procedures
analysis reports 2nd
diagnostic reports 2nd
overview 2nd 3rd
snapshot reports 2nd
reports
analysis reports 2nd
diagnostic reports 2nd
reporting stored procedures
overview 2nd 3rd
snapshot reports 2nd
request-response model
overview 2nd
requirements
SQL-NS
responsiveness (generator optimization)
RESTORE command 2nd
restores
instances
overview 2nd
result values
property values
retention ages
choosing 2nd
manual vacuuming
retention ages (data vacuuming). [See also vaccuming data]
retry expirations
delivery protocols 2nd
rich application features (notification applications)
overview 2nd
SQL-NS
root directories
NewInstances, creating 2nd
rule firing
troubleshooting 2nd
rule firing (quantum processing) 2nd 3rd
rule names
user-defined logic
rule-value terms
RuleName property
Condition APIs
rules
debugging
generators, reenabling after 2nd
overview 2nd 3rd
preparing for 2nd 3rd
quantum clocks, setting for 2nd 3rd 4th
with scheduled subscription debugging stored procedures 2nd 3rd 4th
event and scheduled rules interaction 2nd
failures, generators' handling of 2nd
firing
executing 2nd 3rd
preparing for 2nd
running manually
scheduled rules
music store application 2nd 3rd
semantic errors 2nd
syntax errors 2nd
RuleValue class
Run() method
IEventProvider Interface 2nd
IEventScheduledProvider Interface 2nd
MQHostedEventProvider class
ScheduledHostedEventProvider class 2nd 3rd
runtime behaviors
FileSystemWatcherProvider 2nd
runtime error messages
rule generated errors
runtime errors (generators)
troubleshooting 2nd
runtime files
deployment security and 2nd


S

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

Samples directory
scalability
SQL-NS application models 2nd 3rd
user-defined logic
scalability (notification applications)
capacity limitations (input/output systems) 2nd
overview 2nd
subscription matching costs 2nd
scalability features (Enterprise Edition) 2nd
scaling out
SQL-NS servers 2nd
scaling up
SQL-NS servers 2nd
schedule rules
[brkt-]ConditionAction[-brkt] element
scheduled event providers
troubleshooting 2nd
scheduled event providers. 2nd [See also event providers] [See also custom hosted event providers]
scheduled hosted event providers
assemblies, building 2nd
declaring 2nd
operation of 2nd
overview 2nd
ScheduledHostedEventProvider class
Initialize() method 2nd 3rd
overview
Run() method 2nd 3rd
Terminate() method 2nd 3rd
testing 2nd
scheduled rules
event and scheduled rules interaction 2nd
music store application 2nd 3rd
user-defined logic rule processing
scheduled subscription class
overview
scheduled rules 2nd 3rd
schema for subscriptions by genre 2nd
subscription chronicles 2nd
subscription states 2nd
scheduled subscription debugging stored procedures
NSScheduledSubscriptionDetails 2nd
NSScheduledSubscriptionList 2nd
overview
scheduled subscriptions, adding 2nd
scheduled subscriptions
adding for debugging stored procedures 2nd
music store application
music store database 2nd 3rd
testing 2nd 3rd
times zones, troubleshooting 2nd
user-defined logic
scheduled subscriptions. [See also state maintenance]
ScheduledHostedEventProvider class
Initialize() method 2nd 3rd
overview
Run() method 2nd 3rd
Terminate() method 2nd 3rd
ScheduleRecurrence properties 2nd
ScheduleStart properties 2nd 3rd 4th
schema
user-schema separations 2nd
schema qualified names
music store application 2nd
schema-qualified names
SchemaFile argument
schemas
as namespaces
event-triggered subscriptions 2nd
FileSystemWatcherProvider 2nd 3rd
for events
for notifications
for subscriptions
music store database 2nd
notification schemas
application development process 2nd
SQL-annotated schemas 2nd
subscriptions by genre 2nd
XSD schemas
schemas. 2nd [See also ADF [NPN]] [See also ADFs (Application Definition Files)]
scoping events
scoping of data
scripts
deployment security and 2nd
Scripts directories
setenv.cmd file 2nd
Scripts subdirectories 2nd
SecondaryGroup filegroup
security
content formatters 2nd
event providers
argument encryption 2nd 3rd 4th
argument keys 2nd 3rd 4th 5th 6th
database roles 2nd
overview 2nd
user accounts
development process 2nd
user-defined logic
overview 2nd 3rd
sample application 2nd 3rd
security (notification applications)
overview 2nd
security policies (deployment security)
security sandboxes
user-defined logic security
security violations
exceed throttles, troubleshooting
security vulnerabilities (deployment configurations). [See deployment security]
SELECT clauses
purpose of
SELECT permissions
user-defined logic security
semantic errors
in rules 2nd
separate physical disks
separate physical disks (database systems)
Server class
legacy support
server farms
server login
user-defined logic security 2nd 3rd
ServerConnection class
AddSubscriptions program
StockBrokerNMO sample
ServerInstance property
StockBrokerNMO sample
service accounts
creating 2nd 3rd
setenv.cmd
development environments, testing
environment variables, setting up 2nd
setenv.cmd files 2nd
setup (SQL-NS)
on deployment machines 2nd
setup all.cmd
setup wizard
installing SQL Server components 2nd
shema-qualified names
[brkt-]Database[-brkt] element
signed assemblies
deployment security
simple leaf conditions
simple recovery models (database systems) 2nd
SimpleLeafCondition property
SimpleOperator enumeration
single machine deployment
overview 2nd
skeleton ADFs 2nd
adding code to 2nd
compiling 2nd 3rd 4th
skipped quantums
SMI
subscription schedules, restricting
SMI (Subscription Management Interface)
condition-based logic
SMIs
deployment process 2nd
sample applications. [See AddSubscriptions program]
separating from event providers 2nd
user-defined logic
Condition APIs 2nd
Condition APIs. [See also Condition APIs]
overview 2nd
SMIs (Subscription Management Interfaces)
administrative functions
architecture of 2nd
data layer
connecting to the logic layer 2nd
description of 2nd
logic layer
codes 2nd
connecting to the data layer 2nd
description of 2nd
implementing 2nd
operations 2nd
purpose of 2nd
membership providers (ASP.NET)
database objects, creating for 2nd
MusicStoreSMIOperations class
Genres property 2nd
GetNewSongByArtistSubscriptions() function 2nd
GetNewSongByGenreSubscriptions() function 2nd
overview
SubscriberExists() function 2nd
Subscribers property 2nd
TimeZones property 2nd
overview 2nd
subscriber device operations
adding subscriber devices 2nd
overview
removing subscriber devices 2nd 3rd
subscriber operations
overview 2nd
subscriber records
modifying 2nd
overview 2nd
removing 2nd 3rd
Subscription class
Add() method 2nd
AddNewSongByArtistSubscription() method 2nd 3rd
AddNewSongByGenreSubscription() method 2nd
Delete() method 2nd
FormatScheduleStart() method 2nd
index operator ([])
overview 2nd
ScheduleRecurrence properties 2nd
ScheduleStart properties 2nd
TimeZoneId property
ToString() method
Update() method 2nd
types of 2nd
user-interface layer
description of
Visual Studio solutions
overview 2nd
SMIs (subscription management interfaces)
discontinued use of
SMIs (Subscription Management Interfaces). 2nd 3rd [See also Subscriber Management APIs] [See also Subscriber Management APIs [NPN]] [See also user interfaces [NPN]]
SMO
AddSubscriptions program
SMO APIs
AddSubscriptions program
SMO classes
obtaining
AddSubscriptions program 2nd
obtaining for
AddSubscriptions program
SMOs
reflections (NMO) 2nd
SMOs (SQL Server Management Objects). 2nd [See also NMO]
SMTP delivery channels
creating 2nd 3rd 4th 5th
SMTP delivery protocol
utilizing 2nd
SMTP delivery protocols
declaring
support in the Notification classes 2nd 3rd
testing 2nd 3rd
SMTP delivery protocols. [See also delivery protocols]
SMTP Service
installing 2nd
SmtpServer argument
local machine names, specifying
snapshot reports 2nd
Solution Explorere window (Management Studio)
SongAlerts application
connecting AddSubscriptions program to 2nd
source code
installing 2nd
source code base directories 2nd
source code controls
backing up
source codes
deployment security and 2nd
source files (development environments), customizing 2nd
spam
versus publisher-subscriber models
SQL Client components
installing on deployment machines
SQL compiler
development process
SQL connectivity errors
event submission, troubleshooting
SQL Engine components
installing on deployment machines
SQL errors
event submission 2nd
SQL expressions
computed fields 2nd
SQL joins
matching with 2nd 3rd
SQL passwords
hosted engines 2nd
SQL Server
seperating from the engine 2nd
unable to connect/log in errors 2nd
SQL Server 2003 Notificiation Services. [See SQL-NS]
SQL Server 2005 Management Studio
development process 2nd
SQL Server 2005 Notification Services. [See also Notification Services]
SQL Server Agent
cleanup jobs, setting up
SQL Server Authentication
deployment security
hosted engines 2nd
unable to connect/login errors
SQL Server authentications
development accounts 2nd
development process 2nd
setting up 2nd
test accounts 2nd
SQL Server Configuration Manager
instances, starting 2nd
SQL Server Event Provider. [See SQLProvider]
SQL Server Management Objects (SMO). [See SMOs]
SQL Server Management Objects. [See SMOs]
SQL Server program group
contents of
SQL Servers
various meanings of term
versus development machines
SQL statements
[brkt-]SQLExpression[-brkt] element
overview 2nd 3rd 4th 5th
SQL usernames
hosted engines 2nd
SQL usernames and passwords
deployment security 2nd
SQL-annotated schemas 2nd
SQL-NS
2005 new features 2nd
description of
features overview 2nd 3rd
non-starts, troubleshooting 2nd
notification applications
build benefits 2nd
declarative programming frameworks
engine execution
execution engines
framework extensibility
rich application features
speed of development
official website 2nd
SQL-NS API. [See Subscriber Management APIs]
SQL-NS application models
user-defined logic. [See user-defined logic]
SQL-NS applications
overview
parts of 2nd
SQL-NS applications. [See also application development process]
SQL-NS assembly
browsing for
overview 2nd
referencing
SQL-NS assembly. [See also Subscriber Management APIs]
SQL-NS base diretories. [See source code base directories]
SQL-NS Books Online
class reference
performance objects
subscriber locales
SQL-NS compiler
parameters
SQL-NS compilers
ADFs elements
ICFs (Instance Configuration Files)
SQL-NS condition models
user-defined logic
overview 2nd
SQL-NS engine
distributors
overview 2nd
hosted event providers. 2nd [See also event providers]
SQL-NS engines
seperating from the server 2nd
SQL-NS engines. [See engines]
SQL-NS execution engine
multiple server instance deployment
SQL-NS execution engines
component configuration purpose
SQL-NS installation
on deployment machines 2nd
SQL-NS licensing FAQ
SQL-NS namespace
Subscription Management API classes 2nd
SQL-NS public newsgroup
SQL-NS Redistributable Package
Client Components, installing
SQL-NS services
automatic startup
SqlCommand objects
scheduled hosted event providers
SQLPermissions.sql
SQLProvider
[brkt-]Arguments[-brkt] element
[brkt-]HostedProvider[-brkt] declaration
[brkt-]HostedProvider[-brkt] element 2nd
[brkt-]Interval[-brkt] element 2nd
[brkt-]Schedule[-brkt] element 2nd
[brkt-]StartTime[-brkt] element
chornicles 2nd
events queries 2nd 3rd
overview 2nd
PostQuery arguments
testing 2nd
troubleshooting
versus stored event submission procedures
standalone event providers
advantages of 2nd
building 2nd
configuring in ADFs 2nd
event submission
from applications 2nd 3rd 4th
with stored procedures 2nd 3rd 4th
overview 2nd
XML event providers
code for 2nd
design of 2nd
LoadXml() method
Main() method 2nd 3rd
overview 2nd
testing 2nd
ValidateArguments() method 2nd
Standard versus Enterprise Editions 2nd
start engine programmatically (EngineHost program)
StartCollectionTime column (event batch tables)
starting services (deployment process) 2nd
StartInstance() method
hosted engines 2nd
state mainetenance
event chronicles
declaring 2nd
overview 2nd
schema qualified names 2nd
testing 2nd
event chronicles rules
overview 2nd
overview 2nd
subscription chronicles
music store application 2nd
subscription states
music store application 2nd
statement orders
in property rules)
StockBrokerNMO
Hosting APIs 2nd 3rd
StockBrokerNMO sample
overview 2nd 3rd
testing 2nd
StockBrokerNMO sample. [See also NMO]
StockWatcher application, defining. [See also NMO]
StockWatcher application. [See notification applications]
StopInstance() method
hosted engines 2nd
stored event submission procedures 2nd 3rd 4th
stored procedures
debugging
reporting stored procedures
analysis reports 2nd
diagnostic reports 2nd
overview 2nd 3rd
snapshot reports 2nd
submitting events with 2nd 3rd 4th
strict event batch ordering 2nd 3rd
StringDictionary classes
strong names
installing in Global Assembly Cache (GAC)
strong names (deployment security)
submission (event). [See event submission]
submissionThread variables (continuous hosted event providers)
submitEvents variable
subquantum ordering. [See also strict event batch ordering]
Subscriber class (Subscriber Management API) 2nd 3rd
subscriber data 2nd
subscriber device names
troubleshooting 2nd
subscriber device operations (SMI logic layer)
adding subscriber devices 2nd
overview
removing subscriber devices 2nd 3rd
subscriber device table
subscriber devices
adding for delivery channels 2nd
Subscriber Management APIs 2nd
troubleshooitng
incorret configurations 2nd
troubleshooting 2nd
user-defined logic
subscriber devices names
troubleshooitng
incorret match rule insertion of 2nd
subscriber locale tables
subscriber locales
user-defined logic
Subscriber Management API
classes 2nd 3rd
setting up
overview 2nd
SQL-NS assembly
overview
Subscriber Management APIs
data permission (subscription management) 2nd 3rd
fundamental entities in 2nd
overview
SQL-NS assembly
browsing for
overview
referencing 2nd
SQL-NS instance, preparing 2nd
Subscriber Management APIs. [See also SMIs (Subscription Management Interfaces) [NPN]]
subscriber operations (SMI logic layer)
overview 2nd
records
modifying 2nd
overview 2nd
removing 2nd 3rd
SubscriberDevice class (Subscriber Management API)
SubscriberDeviceEnumeration class (Subscriber Management API)
SubscriberEnumeration class (Subscriber Management API) 2nd
SubscriberEnumeration classes 2nd
SubscriberEnumeration classes. [See also MusicStoreSMIOperations class]
SubscriberExists() function 2nd
SubscriberID property ( SMIs logic layer) 2nd
subscribers
adding
with T-SQL scripts 2nd
device/locale specific subscribers
adding 2nd
Subscriber Management APIs
Subscribers property (MusicStoreSMIOperations class) 2nd
subscribers. [See subscriptions]
subscribtions
device/locale specific subscribtions
adding 2nd
subscripiton classes
defining
for user-defined logic 2nd
subscription chronicles
cleanup 2nd
music store application 2nd
overview 2nd
Subscription class
Condition APIs
generality of
Subscription class (SMI logic layer)
Add() method 2nd
AddNewSongByArtistSubscription() method 2nd 3rd
AddNewSongByGenreSubscription() method 2nd
Delete() method 2nd
FormatScheduleStart() method 2nd
index operator ([])
overview 2nd
ScheduleRecurrence properties 2nd
ScheduleStart properties 2nd
TimeZoneId property
ToString() method
Update() method 2nd
versus subscription classes
Subscription class (Subscriber Management API)
subscription class schemas
device/locale information, storing 2nd
subscription classes
enumerating
with NMO 2nd 3rd 4th
event-triggered subscriptions
declaring 2nd
user-defined logic support 2nd
overview 2nd
versus Subscription class
subscription conditions. [See also conditions]
subscription data
normalizing 2nd
restoring
subscription management API
notification application phases
Subscription Management APIs
logic layer (SMI) wrapping
Subscription Management Interfaces, [See SMIs (Subscription Management Interfaces) [NPN]]
subscription matching 2nd
trigger events, determining
subscription matching costs (notification applications) 2nd
subscription objects
creating
for AddSubscriptions program 2nd
with compound conditions 2nd 3rd 4th 5th
with simple conditions 2nd 3rd
subscription quantum limits. [See also quantum limits]
subscription schedules
restoring
subscription schemas 2nd
subscription selectivity. [See also scheduled subscriptions [NPN]]
subscription states
music store application
overview 2nd
subscription systems. [See also publish-subscribe models]
subscription throttles
exceed throttles, troubleshooting 2nd
subscription throttles (deployment security) 2nd 3rd 4th
subscription view
user-defined logic rule processing
SubscriptionClass class
StockBrokerNMO
SubscriptionEnumeration class (Subscriber Management API)
SubscriptionEventRule class
StockBrokerNMO
SubscriptionField class
StockBrokerNMO
SubscriptionLoader tool 2nd 3rd
SubscriptionLoader. [See also NMO]
subscriptions
adding
in music store application 2nd
as data 2nd
definition of. 2nd [See also publish-subscribe models] [See also events]
displaying
in music store application 2nd
enabling/disabling/removing
in music store application 2nd 3rd 4th
event and scheduled rules interaction 2nd
event-triggered subscriptions
adding 2nd
declaring classes 2nd
music store database 2nd 3rd
overview 2nd
schemas 2nd
event-triggered subscriptions. [See also event triggered matching logic]
granularity of 2nd
management notification application phase 2nd
quantum ordering 2nd 3rd
quantum processing 2nd
retrieving 2nd
scheduled subscription class
overview
scheduled rules 2nd 3rd
schema for subscriptions by genre 2nd
subscription chronicles 2nd
subscription states 2nd
scheduled subscriptions
music store application
music store database 2nd 3rd
testing 2nd 3rd
scoping of data
strict event batch ordering 2nd 3rd
Subscriber Management APIs
Subscriber Management APIs. [See Subscriber Management APIs]
subscribers
adding with T-SQL scripts 2nd
times zones, troubleshooting 2nd
troubleshooting
no subscriptions to process 2nd
non-matching subscriptions and events 2nd
subscriptions table
subscriptions view
match rules
subscriptions. [See also state maintenance]
supllementary ADFs, utilizing
supplementary ADF
supplementary ADFs
music store application
syntax checking
syntax errors
in rules 2nd
syntax highlighting
sysadmin fixed server role
sysadmin fixed server roles
development process
system configuration check
installing SQL Server components


T

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

T-SQL
filegroups, creating 2nd
T-SQL editors
T-SQL scripts
events, submitting
subscribers/subscriptions, adding 2nd
Table class
AddSubscriptions program
TableInputType class
property rules
tables
intermediate tables
joins with 2nd
target devices (notification applications
target users, identifying
tempdb
separating and sizing 2nd
Terminate() method
IEventProvider Interface 2nd
MQHostedEventProvider class 2nd
MQHostedEventProvider class [Aaron-don't connect]
ScheduledHostedEventProvider class 2nd 3rd
terminating variables (continuous hosted event providers)
terms (condition expression)
overview 2nd
test accounts
setting up 2nd
Test.sql
troubleshooting
testenv.cmd
development environments, testing 2nd 3rd
thread pool sizes
distributor optimization 2nd
thread pools
distributor optimization 2nd
Enterprise Edition support
threads
three-tier architecture model (SMIs)
overview 2nd
three-tier architecture model (SMIs). 2nd [See also Subscriber Management APIs] [See also individual layers]
throttles
exceed throttles, troubleshooting 2nd
throttles (deployment security) 2nd 3rd 4th
throughput
user-defined logic
time zones, troubleshooting 2nd
timeouts
troubleshooting 2nd 3rd
while reading notification data 2nd
while updating delivery status 2nd
TimeSpan objects
StockBrokerNMO
TimeZone class (Subscriber Management API) 2nd
TimeZoneEnumeration class (Subscriber Management API) 2nd
TimeZoneId property
TimeZones property (MusicStoreSMIOperations) 2nd
timezones tables
Tools directory
ToolTips
ToString() method
triggering event classes
[brkt-]EventClassName[-brkt] element
user-defined logic
triggers
stored procedures
troubleshooting
additional resources for 2nd
compilation/update errors
malformed XML 2nd
overview 2nd
parameter values, missing 2nd
permission denied errors 2nd
SQL errors 2nd
unable to connect/log in errors 2nd
XML schema compliance errors 2nd
distributors
batch files, failure generating 2nd
content formatters/delivery protocols/non-starting 2nd
device names 2nd
duplicate notification delivery 2nd
duplicate notification generation 2nd
generator fall-behind 2nd
incomplete distributor work items 2nd
non-enabled distributors 2nd
non-starting distributors 2nd
notification expirations 2nd
SQL-NS services, non-starting
subscribe device names 2nd
subscriber device configurations 2nd
subscriber devices 2nd
time zones 2nd
timeouts 2nd 3rd
event providers
exeternal event sources 2nd
incorrect shedules 2nd
non-enabled providers 2nd
runtime errors 2nd
SQL-NS non-starts 2nd
unable to start 2nd 3rd
event submission
SQL errors during 2nd
generators
exceeded throttles 2nd
fall-behind 2nd
no events to process 2nd
no subscriptions to process 2nd
non-enabled generators 2nd
non-matching subscriptions and events 2nd
rule timeouts 2nd 3rd
runtime errors 2nd
overview 2nd


U

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

udate.cmd
versus update_with_argument_key.cmd
UnregisterLocal() method
NMOs
Update Instance command 2nd
troubleshooting 2nd 3rd 4th
Update Instance command (Management Studio) 2nd 3rd 4th 5th
Update() method
subscriber devices
subscriber objects
subscription operations 2nd
update_with_argument_key.cmd
versus update.cmd
updating
instances
overview 2nd 3rd 4th
using Management Studio 2nd
using nscontrol update 2nd
user accounts
security
development process 2nd
service accounts
creating 2nd 3rd
setting up 2nd 3rd 4th 5th 6th
user-defined logic security 2nd 3rd
user interfaces
building 2nd
examing 2nd
pages, initializing 2nd
running 2nd 3rd
subscriptions, adding 2nd
subscriptions, displaying 2nd
subscriptions, enabling/disabling/removing 2nd 3rd 4th
technology options 2nd
user interfaces. [See also SMIs (Subscription Management Interfaces) [NPN]]
user names
security
user-defined logic
[brkt-]ConditionAction[-brkt] element
[brkt-]SqlExpression[-brkt] element 2nd 3rd 4th
implementing 2nd 3rd
internal structure of 2nd
rule processing 2nd 3rd
within [brkt-]EventRule[-brkt] element 2nd
within [brkt-]ScheduleRule[-brkt] element 2nd
AddSubscriptions program
compound conditions 2nd 3rd 4th 5th
expression values, using 2nd 3rd
filter rules and link leaf conditions, creating 2nd 3rd
function values, using 2nd 3rd
overview 2nd 3rd
property rules, creating 2nd 3rd
simple conditions 2nd 3rd
SMO classes, obtaining 2nd 3rd
SongAlerts application, connecting to 2nd
subscription objects, creating 2nd
Visual Studio Solution 2nd 3rd
application support
overview 2nd
subscription classes 2nd 3rd
conditions
composition of in Condition APIs 2nd 3rd
overview 2nd 3rd 4th
types of 2nd 3rd
user-defined rules
user-defined rules. [See also user-defined rules]
evaluating with queries 2nd
sample application
input, creating 2nd 3rd
overview 2nd 3rd 4th
security 2nd 3rd
testing 2nd 3rd 4th 5th
scalability
security
overview 2nd 3rd
SMIs
Condition APIs 2nd 3rd
overview 2nd
subscription classes
defining 2nd
versus developer-defined logic 2nd 3rd 4th
user-defined matching logic
overview 2nd
user-defined rules
filter rules
overview 2nd 3rd 4th 5th
overview 2nd
property rules
overview 2nd 3rd
user-interface layer (SMIs)
description of 2nd
user-schema separations 2nd
usernames
deployment security 2nd
registering instance
UTC times
vacuum schedules 2nd
UTF-8 encoding


V

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

vaccuming
manually invoking 2nd
vaccuuming
quantums and
vacuum schedules
overlapping
vacuum schedules. [See also vacuuming data]
vacuuming data
advantages of 2nd
appropriate settings, choosing 2nd
configuring in the ADF 2nd
overview 2nd
vacuums
enabling
in Management Studio
ValidateArguments() method
XML event providers 2nd
value classes
Condition APIs
verbose informational messages
version control systems
backing up
view materialization (quantum processing) 2nd
View node (Objects Explorer)
ViewInputType class
property rules
Views class
AddSubscriptions program
virtual server names
Visual Studio .NET 2005
development process 2nd
installing 2nd
Visual Studio Solution
AddSubscriptions program 2nd 3rd
Visual Studio solution
Hosting APIs
user interfaces, running
Visual Studio solutions
Hosting APIs
Visual Studio solutions (Subscriber Management APIs)
overview 2nd
Visual Studio. [See also ASP.NET [NPN]]


W

Previous Page
Next Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

warnings
monitoring tasks
WatchDirectory argument
websites
SQL-NS 2nd
WHERE clauses
purpose of
while loops
Windows applications
user interfaces, building
Windows Authentication
deployment security
hosted engines 2nd
unable to connect/login errors
Windows credentials. [See also Windows Authentication]
Windows Data Protection APIs
Windows Server authentications
development process 2nd
setting up 2nd 3rd 4th 5th
Windows Service
configuration database information 2nd
ICFs (Instance Configuration Files)
Windows service. [See also Hosting APIs [NPN]]
Windows services
Create Windows Service check box
instances, starting 2nd
troubleshooting
Windows username and password
specifying
Write() method
Event Object APIs 2nd
MQHostedEventProvider class


X

Previous Page

Index

[SYMBOL] [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X]

XML
[brkt-]Application[-brkt] elements
instances
defining compared to NMO instances defining
malformed XML, troubleshooting 2nd
well-formed XML
xml
lan attributes
XML Bulkload Event APIs
overview 2nd
XML Bulkloader event submission API. [See also XML event providers]
XML editors
development process 2nd
XML elements
ICFs (Instance Configuration Files)
overview
ICFs (Instance Configuration Files). [See also ICFs (Instance Configuration Files)]
XML event providers
code for 2nd
design of 2nd
LoadXml() method
Main() method 2nd 3rd
overview 2nd
testing 2nd
ValidateArguments() method 2nd
XML fragments (XsltFormatters)
XML schemas
compliance errors, troubleshooting 2nd
XSD
duration data types 2nd
XSD schemas
troubleshooting 2nd
XSD validation
XSL transforms
changing content while running
input XMLs 2nd 3rd
locale/device-specific XSL transforms
directory layouts 2nd
music store application 2nd 3rd
overview
testing 2nd 3rd
pass-through XSL transforms
writing for XsltFormatter 2nd 3rd
XSL transforms. [See also content formatters [NPN]]
XsltBaseDirectoryPath arguments
XsltFileName arguments 2nd
XsltFormatter
application development process 2nd
backing up instances
declaring in the ADF 2nd
overview 2nd 3rd
pass-through XSL transforms
XSL transforms
changing content while running
input XMLs 2nd 3rd
locale/device-specific XSL transforms 2nd 3rd 4th 5th 6th 7th 8th
writing for 2nd 3rd
XsltFormatter. [See also content formatters]